Qodo’s fully autonomous agent addresses the complexities of regression testing

Code is constantly evolving in the software development process, requiring continuous testing to verify its quality and maintainability. This is the root of regression testing, where existing tests are re-run to ensure that modified code continues to work as expected.

However, regression testing can be complex and time-consuming and can often be overlooked in lieu of other priorities.

Dig (formerly CodiumAI) says it can alleviate process headaches with the release today of its new fully autonomous AI regression testing agent, Dig cover. Its agent creates validation suites to ensure that software applications, essentially, behave correctly. The 2 and a half year old startup has announced its new tool on AWS re:Inventwhere he was also a finalist in an AWS Unicorn Tank competition.

“We are rapidly approaching a point where the vast majority of code will be generated by artificial intelligence, fundamentally changing the way software is built,” said Itamar Friedman, CEO of Qodo. “It is critical to keep pace by leveraging AI not only to generate code, but also to maintain and improve code quality.”

Keep only code that meets specific criteria

Dig explained earlier this year to VentureBeat Transformation which is approaching AI agents incrementally, taking on competitors such as Devin that offer multiple end-to-end suites. The Israeli startup offers numerous small agents that handle specific tasks within software development workflows.

Qodo Cover is the most recent of these. THE fully autonomous agent analyzes source code and runs regression tests to validate it as it changes throughout its lifecycle. The platform ensures that each test runs successfully, passes and increases the amount of code covered, and only retains those that meet all three criteria.

Demonstrating its ability to generate production quality tests, a self-generated pull request from Qodo Cover was recently accepted on the Hugging Face site PyTorch image template repository. Pull requests are a means of quality control in software development, allowing collaborators to propose and review changes before they are integrated into a code base. This can keep bad code and bugs out of the main code base to ensure quality and consistency.

Hugging Face’s acceptance validates Qodo’s offering and exposes it to more than 40,000 projects in the popular machine learning (ML) repository.

“Qodo Cover represents a significant step towards autonomous software development by ensuring that every piece of code, whether human or not Written by artificial intelligenceit is properly tested and maintainable,” Friedman said.

Qodo Cover is based on an open source project launched by Qodo in May. That project was based on TestGen LLMa tool developed by Meta researchers to fully automate test coverage. To overcome the challenges with tests generated by the Large Language Model (LLM), the researchers decided to answer specific questions:

• Is the test compiled and executed correctly?
• Does the test increase code coverage?

Once these questions have been validated, it is important to perform a manual investigation, Friedman writes in a blog article. This involves asking:

• How well written is the test?
• How much value does it actually add?
• Does it meet any additional requirements?

Users provide various inputs to Qodo Cover, including:

• The source file of the code to be tested
• Existing test suite
• Coverage report
• Command to create and run suites
• Code coverage goals and maximum number of iterations to run
• More context options and suggestions

Qodo Cover then generates multiple tests in the same style, validates them using the runtime environment (i.e., are they created and passed?), examines metrics such as increased code coverage, and updates existing test suites and coverage reports. This operation is repeated until the code reaches the coverage threshold or the maximum number of iterations.

Giving developers full control, it provides progress reports

Qodo’s agent can be deployed as a comprehensive tool that analyzes complete repositories to identify gaps and irregularities and extend test suites. Or it can be established as a GitHub action that automatically creates pull requests to suggest tests for newly modified code. Qodo emphasizes that developers retain full control and have the ability to selectively review and accept tests. Each pull request also includes detailed coverage progress reports.

Qodo Cover supports all popular AI models, including GPT-4o and Claude 3.5 Sonnet. The company claims to provide high-quality results in more than a dozen programming languages ​​including JavaScript, TypeScript, C++, C#, Ruby, Go, and Rust. It is intended to integrate with Qodo Merge, which reviews and manages pull requests, and the Qodo Gen coding tool.