What is GitHub Copilot code review?

Having been around in our IDEs for nearly two years (or closer to three if you include the technical preview), GitHub Copilot has rolled out its latest feature: GitHub Copilot code review, our new friendly neighborhood code reviewer. This feature provides rapid, AI-generated feedback on code changes. It assists us in refining our updates by spotting issues and correcting typos, all with the aim of preparing the pull request for merging before a human reviewer gives the final approval.

Please note that GitHub Copilot code review does not guarantee the detection of all code issues. This caution is clearly stated in the GitHub.com documentation:

Copilot isn't guaranteed to spot all problems or issues in a pull request, and sometimes it will make mistakes. Always validate Copilot's feedback carefully, and supplement Copilot's feedback with a human review.

Currently, the new feature supports the following languages:

• C#

• Go

• Java

• JavaScript

• Markdown

• Python

• Ruby

• TypeScript

You can use GitHub Copilot code review in Visual Studio Code for an initial assessment of a highlighted section and to review your changes, or on GitHub.com for a more in-depth review of the code changes. In this discussion, I will focus exclusively on the review process available on the GitHub website.

How to enable GitHub Copilot code review

There are multiple ways to enable a code review from Copilot. The simplest method is to manually assign Copilot to the pull request, which initiates the code review process:

Manually assigning Copilot to every pull request can become tedious. Fortunately, there's a more efficient solution: enabling automatic code reviews by Copilot. This can be set up through branch rulesets on either a single GitHub repository or specific repositories on organization level. You'll find the "Request pull request review from Copilot" option within the "Require a pull request before merging" checkbox. The workflow is also documented here for your convenience.

Another approach that I find quite effective is managing the ruleset through code and then importing it into the repository or organization. For a single repository, a ruleset that mandates a Copilot code review for every pull request on the default branch would be structured as follows:

{
  "id": 4111833,
  "name": "enable-copilot-code-review",
  "target": "branch",
  "source_type": "Repository",
  "source": "philwelz/github-rulesets",
  "enforcement": "active",
  "conditions": {
    "ref_name": {
      "exclude": [],
      "include": [
        "~DEFAULT_BRANCH"
      ]
    }
  },
  "rules": [
    {
      "type": "pull_request",
      "parameters": {
        "required_approving_review_count": 0,
        "dismiss_stale_reviews_on_push": false,
        "required_reviewers": [],
        "require_code_owner_review": false,
        "require_last_push_approval": false,
        "required_review_thread_resolution": true,
        "automatic_copilot_code_review_enabled": true,
        "allowed_merge_methods": [
          "squash"
        ]
      }
    }
  ],
  "bypass_actors": []
}

Here is an example of a ruleset at the organization level that ensures every pull request on the default branch of all public repos undergoes a Copilot code review.

I hope that in the future, there will be an option to import rulesets using the CLI or GitHub API, allowing for a policy-as-code-like workflow, rather than having to manually import the rulesets through the UI.

Code review in action

Once we've enabled Copilot code review or manually assigned Copilot to a pull request, we're ready to begin.

If the files changed in the pull request are in a supported language, Copilot will review them and check for potential code issues or typos. In my initial example, it identified a typo in the README.md file:

I can now either commit the suggested changes directly to my branch or validate them by opening Copilot in my workspace for further testing. I chose to commit the changes, and once the conversation was resolved, I received this neat PR overview:

If your code changes are in languages that aren't supported, such as a Terraform and Terraform provider updates, you can expect a really simplified output from Copilot:

Bonus: Although GitHub Actions aren't listed in the supported languages section, it appears that Copilot has a better understanding of changes made to them compared to something like Terraform:

Conclusion

GitHub Copilot code review is an exciting enhancement to the developer's toolkit, offering AI-generated insights to improve code quality and streamline the review process. It provides valuable feedback on supported languages, helping to identify potential issues early and enhance the overall efficiency of code reviews. By integrating Copilot code reviews into the standard CI/CD workflow, I anticipate that the traditional "four-eyes" principle will evolve into a "four-eyes-and-AI" principle. However, it's essential to remember that it doesn't replace human judgment. I'm eager to conduct more in-depth tests with a supported language and will be sharing my findings here.

What is LibreChat?

LibreChat is a free and open-source AI chat platform created by Danny Avila. With contributions from over 200 developers, the project continues to gain momentum and has even outlined an roadmap for 2025. Its key design principles focus on delivering a user-friendly interface, supporting a wide range of AI providers, offering extensibility, and prioritizing security. You can find more detailed information about the project here. For the basic setup the following components are needed:

• API/UI: The core application that powers LibreChat's interface and backend.

• MongoDB: A database used to store and handle dynamic, flexible data.

Enhanced features:

• Search: Provides a fast and efficient way to explore and navigate past conversations.

• RAG API: Offers context-aware responses by leveraging user-uploaded files to enhance interactions.

What sets LibreChat apart is its flexibility. The platform can be customized to suit your specific requirements, making it adaptable for a range of use cases and deployments.

For my installation, I decided to proceed with the core application, MongoDB, and optionally enable the search integration, as I currently don’t have a need for the RAG API.

Choosing the right Azure Service(s)

With this decision in mind, I knew I needed to deploy 1-2 applications along with a MongoDB instance. Azure provides a wide range of solutions for hosting web applications, but given my strong passion for cloud-native technologies, containerization was the obvious (and only, let’s be honest) choice for me. However, using Kubernetes felt like overkill for such a lightweight setup, so I choose Azure Container Apps as the home for LibreChat.

Azure Container Apps is a serverless platform specifically designed for scenarios like mine - just provide a container image, configure a few settings, and let the platform handle the rest. It’s a perfect match for this use case!

And then it hit me like a bolt of lightning striking the Hill Valley Courthouse. Azure Container Apps has another feature that’s a perfect fit for this scenario Kubernetes Event-Driven Autoscaling (KEDA). With KEDA, Azure Container Apps can automatically scale HTTP-based applications down to zero when they’re not in use. While this does introduce a brief wait time as the app spins up, it’s completely acceptable for my use case as i dont need the application to be running 24/7. An added bonus? This approach not only saves costs but also reduces environmental impact by lowering CO2 emissions.

The natural choice for deploying MongoDB on Azure is CosmosDB with the MongoAPI (at least for me). While I could have opted to run MongoDB in an Azure Container App, I’m not a big fan of running databases in containers. Instead, I selected the serverless option for the CosmosDB account, allowing me to pay only for what I use—aligning perfectly with the pay-as-you-go model of the container app setup.

CosmosDB provides a lifetime free tier for one CosmosDB account per Azure subscription, making it an appealing option for this setup. However, it's important to note that the free tier does not support the serverless capability.

With the Azure services selected, my go-to tools for deploying this setup are Terraform and GitHub Actions.

Let’s Get Practical: Getting Started

When working with Terraform, one crucial consideration is the management of the state file. The standard practice when using Terraform with Azure is to store the state file in an Azure Storage Account.

The next important aspect is authentication. Historically, Terraform authentication was handled using User Principals or Machine Accounts (Service Principals). However, this approach required storing sensitive credentials and secrets, introducing potential security risks.

In the modern cloud-native world, the de facto standard has shifted to OIDC (OpenID Connect). By leveraging OIDC and Workload Identity Federation, we can utilize managed identities to authenticate and deploy resources with Terraform. This eliminates the need to manage secrets manually and avoids the hassle of creating additional Entra ID (formerly Azure AD) permissions typically required for service principals. It’s a much cleaner, more secure, and practical approach for modern infrastructure-as-code workflows. Check out this blog for a more in-depth exploration of the topic.

To begin the setup, start by cloning the repository and installing the required tools: Azure CLI, GitHub CLI, and jq.

For a faster setup, I’ve created a small script that takes the following inputs. These inputs can be provided either through a .env file or directly within the script:

• name - the project name

• scope - the scope, example: tf for the terraform

• location - the Azure Region to deploy to

• tag - tags for the resources created by the script

• ghRepo - the GitHub repo from where the Terraform code is deployed

Before Azure Resources are created, the script will attempt to retrieve the current Subscription ID and Tenant ID using the Azure CLI. You can customize this behavior to suit your specific requirements.

The script will create the following resources in Azure:

• Resource Group

• Storage Account with Blob Container

• Managed Identity with Owner Role Assignment

• Federated Credentials

• Several GitHub Secrets - for the ease of use in the GitHub Actions

For simplicity, I assigned the managed identity the Owner role on my Azure subscription. However, this approach is not suitable for production environments. In a production-ready setup, you should follow the principle of least privilege and assign only the minimum permissions necessary for your deployment to function correctly.

Let’s take a closer look at the federated credentials.

• The first credential grants the Managed Identity permission to deploy Terraform code, but only from the main branch. This ensures that code deployment from other branches, such as the dev branch, is blocked.

• The second credential provides permission to execute Terraform validation tasks, such as running CI workflows with terraform plan on pull requests.

This setup adds an extra layer of control and security to the deployment process.

# create federated credential for main branch
az identity federated-credential create \
    --resource-group $rg \
    --identity-name "$miName" \
    --name "fc-github-$name-$scope-branch" \
    --issuer "https://token.actions.githubusercontent.com" \
    --subject "repo:$ghRepo::ref:refs/heads/main"\
    --audiences "api://AzureADTokenExchange"

# create federated credential for pull requests
az identity federated-credential create \
    --resource-group $rg \
    --identity-name "$miName" \
    --name "fc-github-$name-$scope-pr" \
    --issuer "https://token.actions.githubusercontent.com" \
    --subject "repo:$ghRepo:pull_request"\
    --audiences "api://AzureADTokenExchange"

Now we are set and done. You can find the complete code for the script in my GitHub repository here.

Up & Running: Final Steps for Your Deployment

Before getting LibreChat up and running, you need to configure an authentication method. I opted to use GitHub as the authentication backend. You can follow the workflow here to set up LibreChat authentication using a GitHub App.

Once the GitHub App is created, you'll need to create the following GitHub secrets:

• LIBRECHAT_APP_ID

• LIBRECHAT_APP_SECRET

Once the desired users have registered, it's important to disable further registrations by setting ALLOW_REGISTRATION = false, as permission management is not implemented. Alternatively, you can choose a different authentication method that better fits your requirements.

Now it's time to configure the AI endpoints. LibreChat supports a wide range of AI providers. The Terraform code available in the repository enables Azure OpenAI and OpenAI by default. If you wish to disable these endpoints and adopt the config with your AI endpoints, you can do so by setting the following variables:

• openai_enabled = false

• azure_openai_enabled = false

The default models for both AI providers are:

• gpt-4o

• gpt-4o-mini

• o3-mini

If you keep the default settings with OpenAI enabled, you'll need to create a GitHub secret named OPENAI_API_KEY and provide your OpenAI Platform API key.

Optional: To enable the Search integration with MeiliSearch, simply set the variable search_enabled to true.

Optional: Custom domain integration. I’m using Cloudflare for managing the custom domain setup. However, you’re free to use your own domain provider or skip this configuration altogether and stick with Azure's default settings.

You can find the configuration options for LibreChat here. Refer to the official documentation for a detailed overview of the available settings.

Once you’ve made the necessary adjustments to the Terraform code and LibreChat config, you’re ready to push it and trigger the deployment using GitHub Actions.

Expanding the Setup: CI and Extras

By this point, all the heavy lifting is complete, and your own GPT should be up and running. Now it’s time to ensure that your Terraform setup and providers are fully up-to-date and to add some Terraform documentation. But don’t worry - I’ve got you covered!

In the repository, you’ll find a workflow for terraform-docs, an action that ensures the Terraform documentation for your code is always up-to-date by automatically committing changes to an open pull request.

But that’s not all. The repository also includes a Renovate workflow, which ensures that your Terraform setup and providers stay up-to-date. Renovate-Bot monitors GitHub releases for updates to these components. Additionally, it keeps MeiliSearch and LibreChat current by tracking new container image releases. The config for Renovate can be found here.

To enable Renovate to monitor GitHub releases, GitHub authentication needs to be configured. This can be achieved using either a Personal Access Token (PAT) or a GitHub App. I chose to use a GitHub App. Once the GitHub App is set up, the following GitHub secrets must be created for use with GitHub Actions:

• RENOVATE_APP_ID

• RENOVATE_PRIVATE_KEY

Last but no least, Labeler. I like labeler because the automation helps streamline workflow organization and makes it easier to quickly identify the purpose of a pull request. As you can see in the example, based on certain changes to files, labels are attached to the pull request.

cicd:
- changed-files:
  - any-glob-to-any-file:
    - .github/**

terraform:
- changed-files:
  - any-glob-to-any-file:
    - src/**

documentation:
- changed-files:
  - any-glob-to-any-file:
    - '**/*.md'

How to get started

You can access the complete Terraform code and GitHub Actions workflows in this GitHub repository. The state of the repository reflects the exact LibreChat GPT version I’m currently using. In other words, this code is live, actively maintained, and subject to regular improvements and changes. It’s a great starting point for setting up your own GPT instance and experimenting with Generative AI. Feel free to dive in, explore, and customize it to suit your needs!

Conclusion

Deploying LibreChat on Azure with Terraform and GitHub Actions is an exciting and practical journey, combining the best of open-source innovation with the flexibility and scalability of cloud-native technologies. By utilizing Azure Container Apps and CosmosDB, you can build a cost-efficient and sustainable setup tailored to your needs. The adoption of OIDC authentication and GitHub Actions simplifies deployment, ensuring a secure and streamlined workflow. Additionally, tools like Renovate and terraform-docs help keep your infrastructure up-to-date and well-documented. This approach not only enables you to operate your own GPT but also embraces modern cloud-deployment best practices, providing a powerful and flexible platform for AI-driven applications.

Due to latest improvements from the Engineering Team that is working on Karpenter at Microsoft, NAP can now be enabled on existing clusters. Follow along to see how your cluster can be migrated to NAP.

Notice: The only supported network configuration to enable NAP on an existing cluster is Azure Overlay with Cilium data plane.

Requirements

As mentioned already NAP is currently public preview, so you need to have Azure CLI with the aks-preview extension 0.5.170 or later installed.

You can verify if you have the extension installed or if you have to correct version with the following commands:

# get installed version of aks-preview:
az extension list | grep aks-preview -A3 -B 3

# if you dont have the extension installed, you can install it with:
az extension install --name aks-preview

# update the extension if dont at least match 0.5.170:
az extension update --name aks-preview

After that, we need to register the NodeAutoProvisioningPreview feature flag:

# register the NodeAutoProvisioningPreview feature flag 
az feature register \
  --namespace "Microsoft.ContainerService" \
  --name "NodeAutoProvisioningPreview"

# verify the registration status
az feature show \
  --namespace "Microsoft.ContainerService" \
  --name "NodeAutoProvisioningPreview"

# re-register the Microsoft.ContainerService resource provider
az provider register --namespace Microsoft.ContainerService

Enable Node autoprovision

To enable NAP on an existing Cluster, simply run this command:

# export this variables as we will will use them more often
CLUSTER_NAME=aks-nap-test-1
CLUSTER_RG=rg-nap-test

# update existing cluster and enable NAP
az aks update \
  --name $CLUSTER_NAME \
  --resource-group $CLUSTER_RG \
  --node-provisioning-mode Auto

After a the update command is successfully finished, we can inspect the Kubernetes API-resources to verify that the Karpenter and the Karpenter Azure provider Custom Resource Definitions (CRDs) were added:

kubectl api-resources | grep -e aksnodeclasses -e nodeclaims -e nodepools

NAME                               SHORTNAMES          APIVERSION                             NAMESPACED   KIND
aksnodeclasses                     aksnc,aksncs        karpenter.azure.com/v1alpha2           false        AKSNodeClass
nodeclaims                                             karpenter.sh/v1beta1                   false        NodeClaim
nodepools                                              karpenter.sh/v1beta1                   false        NodePool

Now we are ready to go. Almost! Before disabling the VMSS node pool(s) we will do some precaution checks and verify if the default NAP resources are added to our cluster (you can also deploy your own NodePools and AksNodeClasses at this step if you dont want to stick with the standard):

kubectl get nodepools

NAME           NODECLASS
default        default
system-surge   system-surge


kubectl get aksnodeclasses

NAME           AGE
default        43s
system-surge   43s

Disable the VMSS node pool(s)

Lets also record the nodes and pods that are running on the cluster. As you can see below we are running a system mode and an user mode AKS nodepool with 3 nodes each and all pods of the inflate deployment are scheduled to the user node pool (as the system node pool has the CriticalAddonsOnly=true:NoSchedule taint):

kubectl get nodes

NAME                                STATUS   ROLES   AGE     VERSION
aks-nodepool1-38290569-vmss000000   Ready    agent   13m     v1.27.7
aks-nodepool1-38290569-vmss000001   Ready    agent   13m     v1.27.7
aks-nodepool1-38290569-vmss000002   Ready    agent   13m     v1.27.7
aks-userpool1-17470761-vmss000000   Ready    agent   4m25s   v1.27.7
aks-userpool1-17470761-vmss000001   Ready    agent   4m25s   v1.27.7
aks-userpool1-17470761-vmss000002   Ready    agent   4m25s   v1.27.7

kubectl get pods -owide

NAME                       READY   STATUS    RESTARTS   AGE     IP             NODE                                NOMINATED NODE   READINESS GATES
inflate-74ccd665f4-7xj2w   1/1     Running   0          6m32s   10.244.5.136   aks-userpool1-17470761-vmss000001   <none>           <none>
inflate-74ccd665f4-bgdjh   1/1     Running   0          114s    10.244.4.212   aks-userpool1-17470761-vmss000000   <none>           <none>
inflate-74ccd665f4-k5mt8   1/1     Running   0          6m32s   10.244.3.128   aks-userpool1-17470761-vmss000002   <none>           <none>

Now we are ready to finally disable the user mode node pool(s) by simply scaling it to 0. We could also decide to leave this static node pool(s) as NAP and Karpenter supports this scenario but wont autoscale them. The system node pool has to stay untouched in this scenario.

If the target node pool(s) have cluster-autoscaler enabled we have to disable it before scaling to 0

# variable for the VMSS node pool name
NODE_POOL=userpool1

# disable cluster-autoscaler before scaling to 0 if enabled
az aks nodepool update \
  --name $NODE_POOL \
  --name $CLUSTER_NAME \
  --resource-group $CLUSTER_RG \
  --disable-cluster-autoscaler

# scale user node pool to 0
az aks nodepool scale \
  --name $NODE_POOL \
  --name $CLUSTER_NAME \
  --resource-group $CLUSTER_RG \
  --no-wait \
  --node-count 0

Now we can let NAP do what NAP does best, choose automatically the best suitable node size for our workload. After 1-2 minutes or workload should be up and running:

kubectl get events -A --field-selector source=karpenter -w

NAMESPACE   LAST SEEN   TYPE     REASON                      OBJECT                         MESSAGE
default     52s         Normal   Nominated                   pod/inflate-74ccd665f4-g2f5b   Pod should schedule on: nodeclaim/default-6hx6j
default     52s         Normal   Nominated                   pod/inflate-74ccd665f4-lt5tf   Pod should schedule on: nodeclaim/default-6hx6j
default     52s         Normal   Nominated                   pod/inflate-74ccd665f4-xbdt6   Pod should schedule on: nodeclaim/default-6hx6j
default     0s          Normal   Unconsolidatable            node/aks-default-6hx6j         Can't replace with a cheaper node

kubectl get nodeclaims

NAME            TYPE               ZONE           NODE                READY   AGE
default-6hx6j   Standard_D4ls_v5   westeurope-2   aks-default-6hx6j   True    5m12s

kubectl get pods -o wide

NAME                       READY   STATUS    RESTARTS   AGE     IP             NODE                NOMINATED NODE   READINESS GATES
inflate-74ccd665f4-g2f5b   1/1     Running   0          5m52s   10.244.3.198   aks-default-6hx6j   <none>           <none>
inflate-74ccd665f4-lt5tf   1/1     Running   0          5m53s   10.244.3.11    aks-default-6hx6j   <none>           <none>
inflate-74ccd665f4-xbdt6   1/1     Running   0          5m53s   10.244.3.2     aks-default-6hx6j   <none>           <none>

Voilà! We enabled Node autoprovisioning on our existing AKS cluster. The unused node pools(s) can now safely be removed if wanted:

# delete unused node pool(s) if wanted
az aks nodepool delete \
  --name $NODE_POOL \
  --name $CLUSTER_NAME \
  --resource-group $CLUSTER_RG \

Final words

Check out the official docs for NAP if you want to have a deeper overview about this new feature. Hopefully the documentation will soon remove the limitation statement that Node autoprovisioning can be only enabled on new clusters. Until that lets spread the word that is possible as of today.

At last I suggest to leave a star for Karpenter Provider for Azure on GitHub. Please also help to improve the Karpenter on Azure provider by submitting feedback of issues while using NAP or work on issues labeled with good first issue.

You can catch me on X or LinkedIn if you want to chat. Cheers.