Software engineers at ANZ Bank saw a "notable boost" in productivity and code quality when using the GitHub Copilot AI tool.
Last year, around 100 engineers at the Australian bank carried out a six week trial of the tool for a study evaluating both their sentiment on using it and its impact on their productivity, code quality and security.
During the study, participants were split into control and Copilot groups, each tackling the same Python challenges.
The group that had access to Copilot was able to complete their tasks 42% faster than the control group participants. In addition, the code produced by Copilot participants contained fewer code smells and bugs on average, meaning it would be more maintainable and less likely to break in production.
However, there was no meaningful data on the impact on code security, although CoPilot did not introduce any security issues.
Participants were also fairly enthusiastic about using the technology, saying it had a "positive effect" on their ability to review and understand existing code, create documentation, and create unit tests.
The paper concludes that the research "provides compelling evidence of the transformative impact of GitHub Copilot on engineering practices at ANZ Bank. The adoption of this tool has marked a shift in the work paradigm, empowering engineers to focus more on creative and design tasks while reducing time spent on repetitive coding."
There is now significant adoption of CoPilot within the bank, with around 1000 users who report an uptick in productivity and job satisfaction.
Read the paper