Scaling ChatGPT to Building Larger Applications with Microtask Programming

Abstract

ChatGPT and other large language models (LLMs) bring the potential to revolutionize programming by enabling developers to synthesize code and whole programs from natural language descriptions. However, current LLMs are limited in the size of the code they can generate and by their tendency to generate incorrect code. These limitations require new ways for human developers to guide the code generation offered by LLMs. To explore this direction, we investigated using LLMs to generate code within a microtask programming workflow. Microtask programming is a software development method in which developers write complex software through short, self-contained microtasks including testing, implementing, debugging, and reviewing functions. We envision using ChatGPT to write code for one function, which developers can then use microtasks to review and to suggest changes, enabling more complex software application to be built step by step. We investigated the feasibility of this process by using ChatGPT to generate code from descriptions of functions taken from microtasks and evaluating their correctness using a unit test suite. Our findings reveal that ChatGPT-generated functions successfully passed 70% (24 out of 34) of unit tests in one codebase and 59% (23 out of 39) in a second codebase. Overall, our findings indicate promising prospects for using ChatGPT as part of a microtask workflow to build more complex software.