Developers, say hello to Coding Assistant – your new best (AI) friend!

Although AI is everywhere, it lacks a formal definition and is often confused with machine learning research and engineering – distinct fields altogether.

Tejas Kumar, Developer Relations Engineer,
DataStax and a Shift conference speaker, offers a different perspective: “AI engineering doesn’t require a background in academia, machine learning models, Python, or linear algebra; it’s simply about applying AI to solve problems.”

According to Kumar, AI can be utilized without training models yourself – just by making a network request to an AI API and leveraging the output to address various challenges.

So, how’s AI going to shake up software development over the next 5-10 years?

Well, language models are already stirring the pot, making it way easier to tackle those tricky new languages and frameworks. As Zvonimir Petkovic (Senior Software Engineer, Infobip) puts it, this shift is just getting started.

“Models are getting better at understanding and working with code, as shown by benchmarks like HumanEval and BigCodeBench. With tools like GPT-3.5-Turbo and the new Claude-3.5-Sonnet, this trend is only going to get bigger!”

But the next big hurdle? Taking on complex tasks like analyzing entire codebases and performing major refactoring, which is still a bit of a wild card.

“Sure, there are cool early projects like Amazon Q, but it’ll take a while for these to become mainstream. Right now, models like Transformers have a hard time with long contexts. That’s where new ideas like Mamba and Jamba come in, offering better ways to get the big picture. More and more developers, whether they’re just starting out or have been in the game for a while, are jumping on board, letting language models handle the repetitive stuff. And as Andrej Karpathy hinted, it might not be long before English itself becomes the new go-to programming language!”

Now you might be wondering, “Forget about the next 5 years – what AI tools are making waves in software development right now?”

According to Zvonimir, it’s Coding Assistants.

“They’re becoming a key tool in the SDLC, expected to take on more tasks as time goes on. Features that don’t need user input, like auto-generating commit messages, are quickly being adopted because they save time on repetitive work. Coding assistants have already shown their value and will continue to be important.”

Plus, new solutions are emerging for things like root cause analysis, log integration, and incorporating domain knowledge and custom instructions – similar to the tech used by AI assistants.”

But it’s not all smooth sailing – AI has its quirks, like hallucinations!

Tejas explains that Retrieval-Augmented Generation (RAG) is a solution to this problem.

“It combines pre-trained models with external knowledge retrieval, while fine-tuning adapts models to specific tasks. RAG helps reduce hallucinations by grounding responses in factual information, whereas fine-tuning can enhance performance on specific tasks but might lead to overfitting.”

Zvonimir adds that overcoming these challenges involves picking up new soft skills known as prompt engineering to work effectively with language models. This includes:

Want to know more about AI? Tejas Kumar and Zvonimir Petković are speaking at this year’s Shift Conference in Zadar. Find out more details here.