This is exactly why I’m starting a consulting practice focused in this area. Similarly, I’m thinking of starting an AI executive coaching service focused on helping individuals empower themselves with AI tools. So much potential ahead!
> A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023.
ChatGPT
The statement is an exaggeration and not entirely true. While advancements in AI technology have made certain tasks more accessible and streamlined, it is not accurate to claim that a wide range of AI tasks that previously required years of research and a dedicated team can now be accomplished easily with just API documentation and a spare afternoon.
AI development still requires a solid understanding of the underlying principles, algorithms, and methodologies. Complex AI tasks, such as training deep learning models or solving intricate problems, still require significant expertise, time, and computational resources. While APIs and pre-trained models can simplify certain aspects of AI development, they do not eliminate the need for expertise and careful implementation.
It's important to recognize that AI development is a rapidly evolving field, and advancements in technology continue to make certain tasks more accessible. However, it is still a complex and specialized area that often requires dedicated time, research, and expertise to achieve meaningful and reliable results.
This is great and thanks for all of the links and references.
FWIW - I agree that the under the hood stuff really isn't necessary for AI engineering in the future because it's not even really necessary now. Yes learning from first principles matters, but most DS and MLE for 10+ years would set up their models with sci-kit learn and understood that they had to optimize for key business metrics, latency and performance, and minimum viable precision / recall metrics rather than understanding the fundamentals of linear algebra and tensorflow to make the VERY best and accurate models from scratch.
And I personally love treating AI LLMs like a black box because it allows this AI engineer role to train a skillset that combines business + product + implementation that I think is almost always more valuable.
Great article! And timely appearance of the AI Engineer Summit! I organize even more niche event online - Prompt Engineering Conference - just two days after the Summit. Free registration and CFP are open!
I thought I'd have to follow links here and there to get the gist of the article like most tech specific articles are but this one was extremely well written !!!
Hey swyx, thanks for putting together a well-articulated "description" of an evolving job - AI Engineer. Out of curiosity, I have been doing these things since ChatGPT came out and once in a while scratching my head wondering: "What kind of job is this?"
Hi @swyx! Extremely informative article - I'm currently working as a Marketing Automation Manager, but more and more tasks I get, the closer they are related to automating something with AI (i.e. by fine-tuning LLM)
As my job is being "influenced" by AI, I am looking to get really deep into AI Eng. Don't know a lot about programming, but previously I've done CS50 course by Harvard.
What path would you recommend for a beginner to start with AI Engineering?
You might need to learn new skill, but no new role is needed. A good engineer is what a company need. There is no need to spend too much time to think about this at all, a good developer will already learn new skills and be relevant.
This is exactly why I’m starting a consulting practice focused in this area. Similarly, I’m thinking of starting an AI executive coaching service focused on helping individuals empower themselves with AI tools. So much potential ahead!
great followup from @jjacky here: https://jjacky.substack.com/p/ai-will-not-replace-you-but-you-will
> A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023.
ChatGPT
The statement is an exaggeration and not entirely true. While advancements in AI technology have made certain tasks more accessible and streamlined, it is not accurate to claim that a wide range of AI tasks that previously required years of research and a dedicated team can now be accomplished easily with just API documentation and a spare afternoon.
AI development still requires a solid understanding of the underlying principles, algorithms, and methodologies. Complex AI tasks, such as training deep learning models or solving intricate problems, still require significant expertise, time, and computational resources. While APIs and pre-trained models can simplify certain aspects of AI development, they do not eliminate the need for expertise and careful implementation.
It's important to recognize that AI development is a rapidly evolving field, and advancements in technology continue to make certain tasks more accessible. However, it is still a complex and specialized area that often requires dedicated time, research, and expertise to achieve meaningful and reliable results.
This is great and thanks for all of the links and references.
FWIW - I agree that the under the hood stuff really isn't necessary for AI engineering in the future because it's not even really necessary now. Yes learning from first principles matters, but most DS and MLE for 10+ years would set up their models with sci-kit learn and understood that they had to optimize for key business metrics, latency and performance, and minimum viable precision / recall metrics rather than understanding the fundamentals of linear algebra and tensorflow to make the VERY best and accurate models from scratch.
And I personally love treating AI LLMs like a black box because it allows this AI engineer role to train a skillset that combines business + product + implementation that I think is almost always more valuable.
Great article! And timely appearance of the AI Engineer Summit! I organize even more niche event online - Prompt Engineering Conference - just two days after the Summit. Free registration and CFP are open!
https://promptengineering.rocks/
@swyx, let's set up a partnership!
This was surprisingly succinct
I thought I'd have to follow links here and there to get the gist of the article like most tech specific articles are but this one was extremely well written !!!
Hey swyx, thanks for putting together a well-articulated "description" of an evolving job - AI Engineer. Out of curiosity, I have been doing these things since ChatGPT came out and once in a while scratching my head wondering: "What kind of job is this?"
I write AI Made Simple. We have very synergistic audiences. Do you want to collab?
Hi @swyx! Extremely informative article - I'm currently working as a Marketing Automation Manager, but more and more tasks I get, the closer they are related to automating something with AI (i.e. by fine-tuning LLM)
As my job is being "influenced" by AI, I am looking to get really deep into AI Eng. Don't know a lot about programming, but previously I've done CS50 course by Harvard.
What path would you recommend for a beginner to start with AI Engineering?
Aren't you focusing too much on LLM and natural language processing? In the end this is just a subset (as you highlighted in the news trends graph)
Grateful to have read this. What are your thoughts on certifications like the Microsoft Azure AI Engineer as a starting point?
You might need to learn new skill, but no new role is needed. A good engineer is what a company need. There is no need to spend too much time to think about this at all, a good developer will already learn new skills and be relevant.