Evaluate the suitability of LLMs for the generation of Infrastructure as Code to provision, configure, and deploy modern applications
In this article, we address the suitability of LLMs to leverage the lifecycle of real applications, ranging from the infrastructure provisioning to the configuration management and deployment. The source code resulting from this effort is publicly available on GitHub¹¹. Infrastructure as Code (IaC) solutions facilitate the management and provisioning of infrastructure through code instead of through a manual process¹. They are becoming commonplace, and major cloud providers have implemented their own flavor of IaC solutions for interacting with their services. In this regard, AWS CloudFormation, Google Cloud Deployment Manager, and Azure Resource Manager Templates streamline the provisioning of cloud services, eliminating the need for IT Operations to manually spin up servers, databases, and networks. However, these many possibilities introduce the risk of vendor locking since the definition of the required IaC for a given cloud provider is not portable and would need to be translated if a different cloud provider is required. In this regard, tools like Terraform² or Pulumi³ provide an abstraction over the various implementations of the different cloud providers and facilitate the development of portable deployments. This way, the risk of vendor locking is severely reduced, and organizations can react dynamically to their needs without incurring significant implementation costs. On top of this, there are numerous benefits that IaC technologies bring to the table⁴:
- Consistency: it permits the automation of the infrastructure provisioning by enabling repeatable deployments.
- Decreased Risk: it promotes a less error-prone approximation to infrastructure management, as manual interventions are minimized.
- Cost Optimization: it enables easier identification of unnecessary resources, and faster migration among cloud providers reacting to billing changes.
- Improved Collaboration: scripts can be integrated into version control tools, which promotes collaboration between individuals and teams.
However, the application lifecycle goes beyond infrastructure provisioning. The following figure displays the application lifecycle supported by different IaC technologies⁵.
In this scenario, the goal of IaC technologies extends beyond the mere provisioning of infrastructural resources. After spinning up the necessary infrastructure, the configuration management stage ensures that all the requirements are appropriately installed. This stage is usually accomplished with tools such as ansible⁶, chef⁷, or puppet⁸. Finally, the application deployment oversees the orchestration of the application over the various infrastructural devices.
Large Language Models (LLMs) refer to a class of artificial intelligence models that are designed to understand and generate human-like text based on the input provided to them. These models are known for their considerable number of parameters, which enable them to capture complex patterns and nuances in language⁹.
- Text Generation: Text created by LLMs can be cohesive and relevant to its surroundings. They are employed for activities like finishing texts, producing material, and even doing creative writing.
- Language Comprehension: LLMs are capable of comprehending and extracting information from text. They are capable of attitude analysis, text classification, and information retrieval.
- Translation: LLMs can translate text from one language to another. This is very beneficial for machine translation applications.
- Answering questions: LLMs can answer questions based on a given context. They are used in chatbots and virtual assistants to answer user queries.
- Text Summarization: LLMs can summarize long passages of text into shorter, more coherent summaries. This is useful for condensing information for quick consumption.
Among the aforementioned capabilities, we will focus on text generation. In particular, in its ability to produce the correct IaC code based on the input prompts, Large Language Models (LLMs) have made significant advances in the field of natural language processing, but they have also raised several challenges and concerns. Some of the key issues and concerns associated with LLMs at that time include:
- Biases and fairness: LLMs can learn biases present in the data they are trained on, which can lead to biased or unfair results.
- Misinformation and disinformation: LLMs may generate false or misleading information, which may contribute to the spread of misinformation and disinformation online. These models have the potential to create content that appears credible but is factually incorrect.
- Security and privacy: LLMs can be misused to generate malicious content, such as deepfake texts, fake news, or phishing emails.
The following table displays a comparison among various LLMs¹⁰
In order to test how current LLM tools perform in the field of IaC, a use case has been designed. The final objective is to build an API in a virtual machine using the FastAPI framework that allows the client to perform search and delete tasks in an Elasticsearch cluster using HTTP methods. The cluster will be composed of three nodes, each one in its own virtual machine, and in another machine will be Kibana, an administration tool for the cluster that supports visualizations. Everything must be in the AWS cloud. The following figure shows this architecture:
The challenge is to complete the following three tasks successfully using LLM tools. In this article, we have used OpenAI’s ChatGPT.
- Terraform code to build an infrastructure with five virtual machines in AWS.
- Source code of the FastAPI application to perform document search and delete operations through HTTP methods of the API.
- Ansible code for the deployment and installation of an Elasticsearch cluster on three nodes, Kibana on another node, and a FastAPI application on the remaining node.
For this challenge, we have used the following prompt:
I need to create, via Terraform, five virtual machines at a public cloud provider you want. The purpose of these virtual machines is the following: Three of them are for deploying an Elasticsearch cluster, which is going to ingest 2G of data each day; the other one is for Kibana; and the last one is for deploying a FastAPI application. You should choose the hardware for each virtual machine and the cloud provider for each virtual machine. For the variables you don’t know, use variables as placeholders. Use cheap instances.
The initial response was a good effort, but we needed to keep iterating. For instance, we wanted all the variables to be defined in a separate file, which resulted in the following image.