In my last blog “Where Do Programmers Fit Into the Age of AI?”, I came to the conclusion that the demand for software developers is not going anywhere as a result of the rollout of AI. The role of programmer itself is in fact evolving and the skills required to be proficient, are changing. That leads to our next logical question: What does the core skillset look like for an effective developer in AI era?
This is where we run into gray area when it comes to how AI is being applied to the process of software development. There is a wide range of use cases from people ‘vibe coding’ all the way to professional developers’ syntax coding and everything that lies in between.
It seems to me that with so many methods of using AI, the best way to figure out what is working well and what isn’t working so well is to measure the outcomes of their application in a tangible way.
After giving it some consideration, I think metrics that measure production capacity, will ultimately be what we are looking at when it come to measuring the performance of LLM’s in particular.
It could come down to things like:
Time to solution: The time it takes to get from realizing a problem exists to the time you have a viable solution that mitigates the issue effectively.
Quality of solution: The effectiveness of the application to solve the initial problem.
Maintainability of solution: The time and effort it takes to maintain a codebase after it’s been implemented.
Quality of query: The structure and content of a query will influence what is provided in response to us by the LLM. Some people will naturally be better at this than others.
Time to query composition: The time it takes to generate an effective query that returns the desired result you were looking for from the outset.
The next important question becomes: Can we learn these skills and if so, can we teach these skills to others? If this is the case it will become a new segment of the workforce that didn’t previously exist prior to the dissemination of LLM’s.
One thing I do know in this new era of AI is that LLM’s are not Artificial General Intelligence (AGI) yet. We need to keep what they are in perceptive because it’s easy to listen to the industry hype, see some interesting things being pumped out of an LLM and think that it’s some sort of witchcraft. When in reality its nothing more than a really good token predictor. They are still just a tool to be used for your own pursuits and we should never lose sight of that.
If your pursuit is education, the old adage still rings true: “Be responsible for your own learning.” and to add to that for the world of misinformation we find ourself living in today “Seek out reliable information”. How we go about this is changing, but the intent remains the same.
Peter Linton
Software Developer
