Install the LLVM 15 Toolchain

Mục tiêu: A step-by-step guide to installing and configuring the LLVM 15 toolchain on various operating systems (macOS, Linux, Windows) as the backend for a compiler project.

Laying the Foundation: Installing the LLVM Toolchain

Welcome to the first step in our exciting journey of building a compiler! Before we can write a single line of Rust code to parse or generate anything, we need to set up the most critical piece of our compiler’s backend: the LLVM toolchain.

What is LLVM and Why Do We Need It?

LLVM, originally the Low Level Virtual Machine, is a collection of modular and reusable compiler and toolchain technologies. Think of it as a powerful “compiler backend as a service.” Our job will be to create the “frontend”—the part that understands our custom SimpliLang syntax. We will translate our code into an intermediate language that LLVM understands, called LLVM Intermediate Representation (IR).

Once we have the LLVM IR, we hand it over to LLVM. It then takes on the heavy lifting of:

  1. Optimizing the code with a vast suite of battle-tested optimization passes.
  2. Generating native machine code for a wide variety of target architectures (x86, ARM, etc.).

By using LLVM, we can focus on the unique aspects of our language’s design and logic, while leveraging a world-class infrastructure for code optimization and generation.

For this project, we’ll be targeting LLVM 15. The Rust crate we’ll use to interact with LLVM, inkwell, has bindings that are tied to a specific LLVM version. Sticking to a single version ensures that the API calls we make from Rust match what the underlying C++ LLVM library expects, preventing compatibility headaches.

Let’s get it installed on your system.

Installation Instructions by Operating System

Please follow the instructions for your specific operating system.

macOS

The most straightforward way to install specific versions of development tools on macOS is with Homebrew.

  1. Install LLVM 15: Open your terminal and run the following command. This will download and install the pre-compiled binaries for LLVM 15.

    bash brew install llvm@15

  2. Configure Environment Variables: Homebrew installs versioned formulas “keg-only,” which means they aren’t automatically added to your system’s PATH to avoid conflicts with the system’s default tools. We need to tell our shell where to find the LLVM 15 binaries and libraries.

    Add the following lines to your shell’s configuration file (e.g., ~/.zshrc for Zsh or ~/.bash_profile for Bash):

    ```bash

    Add LLVM 15 to the PATH

    export PATH=”/opt/homebrew/opt/llvm@15/bin:$PATH”

    For the llvm-sys crate to find the correct LLVM installation

    export LLVM_SYS_150_PREFIX=”/opt/homebrew/opt/llvm@15” ``` ``

    • Explanation:
    • The ```PATHvariable tells your shell where to look for executable programs. By prepending LLVM 15'sbindirectory, you ensure that when you typeclangorllc`, you get the version from LLVM 15, not the system default.
    • TheLLVM\_SYS\_150\_PREFIXvariable is a special instruction for thellvm-sysRust crate (a dependency ofinkwell). It tells the build script exactly where to find the headers and library files for LLVM 15 during compilation. The150 corresponds to version 15.0.

  3. Apply Changes: For the changes to take effect, you must either restart your terminal or run source ~/.zshrc (or the equivalent for your shell).

Linux (Debian/Ubuntu-based)

Linux distributions often have their own LLVM packages, but they might not be the exact version we need. The recommended approach is to use the official LLVM APT repository.

  1. Add the LLVM APT Repository: These commands will download the repository’s GPG key and add the source to your package manager.

    
# Get the key

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -

# Add the repository. Replace 'focal' with your Ubuntu/Debian version codename

# (e.g., 'jammy' for Ubuntu 22.04, 'bullseye' for Debian 11).

sudo add-apt-repository 'deb http://apt.llvm.org/focal/ llvm-toolchain-focal-15 main'

  2. Install LLVM 15 Packages: Now, update your package list and install the necessary components.

    bash sudo apt-get update sudo apt-get install -y llvm-15-dev libclang-15-dev clang-15 * Explanation of Packages: * llvm-15-dev: Provides the core LLVM libraries and header files needed for development. * libclang-15-dev: Provides the libraries for Clang, which inkwell also uses. * clang-15: The Clang C/C++ compiler frontend itself, which we can use later for linking our compiled object files.

    On Linux, the llvm-sys crate is usually smart enough to find the installation using the llvm-config-15 tool, so setting environment variables is often not required.

Linux (Arch-based)

Arch Linux and its derivatives are rolling-release, so the main repository might have a version newer than 15. You might need to use the Arch User Repository (AUR) for a specific version.

  1. Check Official Repositories (First Try): First, check if a suitable version is available. `bash

    This will likely install a newer version, but it’s the simplest way

    sudo pacman -S llvm libclang clang If this installs a version newer than 15, and inkwell` fails to build later, you’ll need the specific version from the AUR.

  2. Install from AUR (Recommended for specific version): Using an AUR helper like yay is the easiest method. bash yay -S llvm15 This will find, build, and install the llvm15 package, which is specifically LLVM version 15.

Windows

Windows setup requires a few more manual steps.

  1. Install LLVM Binaries: Download the pre-built binaries from the official LLVM GitHub Releases. Look for the file named LLVM-15.0.7-win64.exe.
  2. Run the Installer: During the installation process, you will be presented with an options screen. It is very important that you check the box that says “Add LLVM to the system PATH for all users” (or current user). This will make the LLVM tools available on the command line.

  3. Install a C++ Toolchain: LLVM requires a C++ compiler and linker to function. The easiest way to get this is by installing the Build Tools for Visual Studio.

    • Go to the Visual Studio Downloads page.
    • Under “Tools for Visual Studio”, find “Build Tools for Visual Studio” and download the installer.
    • When the installer runs, select the “Desktop development with C++” workload and click Install.

  4. Set Environment Variable (if needed): If the Rust build script fails to find LLVM automatically, you may need to set the LLVM_SYS_150_PREFIX environment variable manually.

    • Press the Windows key and type “Edit the system environment variables”.
    • Click the “Environment Variables…” button.
    • Under “System variables”, click “New…”.
    • Set the Variable name to LLVM_SYS_150_PREFIX.
    • Set the Variable value to the path where you installed LLVM (e.g., C:\Program Files\LLVM).

Verifying the Installation

Once you have completed the steps for your OS, it’s time to verify that everything is working. Open a new terminal window to ensure all environment variable changes have been loaded.

Run the following commands:

# On macOS/Linux, you may need to use the versioned names
llc-15 --version

# On Windows or systems where you've set the PATH, this should work
llc --version

You should see output similar to this, confirming you are using version 15:

LLVM (http://llvm.org/):
  LLVM version 15.0.7
  ...

The tool llc is the LLVM Static Compiler. It’s a key utility that compiles LLVM IR files into object files, and we will use it later in our project. Seeing this output confirms that the LLVM toolchain is installed and accessible from your command line.

Next Steps

Fantastic! You’ve laid the essential groundwork for our compiler. With the LLVM toolchain ready to go, our next task is to create the Rust project that will become our SimpliLang compiler.

Further Reading

Initialize the Rust Compiler Project with Cargo

Mục tiêu: Use Cargo, Rust’s build tool, to create a new binary project for the SimpliLang compiler. This task covers using ‘cargo new’ to scaffold the project, understanding the generated directory structure, and running the initial ‘Hello, world!’ program.

Excellent! With the LLVM toolchain installed and verified, you’ve set up the foundational layer of our compiler’s backend. Now, it’s time to create the Rust project that will become the home for our SimpliLang compiler. This is where we introduce Rust’s indispensable build tool and package manager: Cargo.

Introducing Cargo: Your Rust Project’s Best Friend

In the world of C or C++, you might be used to managing your own build systems with tools like make, CMake, or even just direct compiler commands. The Rust ecosystem simplifies this dramatically with Cargo.

Cargo is the official Rust build tool and package manager. It handles a multitude of tasks for you, including:

  • Project Scaffolding: Creating a standardized directory structure for new projects.
  • Dependency Management: Downloading and compiling the libraries (called “crates” in Rust) your project needs.
  • Building Code: Compiling your source code into a binary executable or a library.
  • Running Tests: Executing your project’s unit and integration tests.
  • And much more…: Publishing libraries, generating documentation, etc.

By using Cargo, we can focus on writing our compiler’s logic instead of getting bogged down in the complexities of build configuration.

Creating the simplilang_compiler Project

Let’s use Cargo to create our new project. Open your terminal and navigate to the directory where you want your projects to live. Then, run the following command:

cargo new simplilang_compiler

Let’s break down this simple command: * cargo: This is the executable for the Cargo tool itself. * new: This is a subcommand that tells Cargo to create a new project. * simplilang_compiler: This is the name we’re giving our project. Cargo will create a new directory with this exact name.

After running the command, you will see a confirmation message:

     Created binary (application) `simplilang_compiler` package

This tells you that Cargo has successfully created a new project configured as a binary application (meaning it will compile to an executable file), as opposed to a library.

Exploring the Project Structure

Now, let’s look at what Cargo created for us. Navigate into the new directory:

cd simplilang_compiler

If you list the contents of this directory (ls -a on Linux/macOS or dir on Windows), you will see the following structure:

.
├── .git
├── .gitignore
├── Cargo.toml
└── src
    └── main.rs

This is the standard layout for a Rust binary project, and understanding it is key:

  • .git/: Cargo automatically initializes a new Git repository for you. Version control from day one is a best practice!
  • .gitignore: A standard .gitignore file is included, pre-configured to ignore Rust’s compilation artifacts, most notably the /target directory where compiled files are placed.
  • src/: This is the directory where all of your Rust source code will live.
  • src/main.rs: For a binary project, this file is the crate root and the entry point of your program. Cargo has already placed a simple “Hello, world!” program inside it for you.
  • Cargo.toml: This is the manifest file for your project. It’s written in the TOML (Tom’s Obvious, Minimal Language) format and contains all the metadata Cargo needs to compile your project.

Let’s look at the initial contents of Cargo.toml:

[package]
name = "simplilang_compiler"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
  • [package] section: Contains metadata about your project.
    • name: The name of your package.
    • version: The version, following the Semantic Versioning (SemVer) standard.
    • edition: The Rust edition to use. “2021” is the latest major edition and enables the newest language features.
  • [dependencies] section: This is where we will list all the external crates (libraries) our project needs. It’s empty for now, but we will add to it in the very next task.

Running the Boilerplate Code

To confirm that your Rust installation and the new project are working correctly, you can use the cargo run command from within the simplilang_compiler directory.

cargo run

This command does two things:

  1. Compile: It invokes the Rust compiler (rustc) to build your project. The first time you run this, it will be a bit slow as it compiles everything. Subsequent runs will be much faster because Cargo caches build artifacts and only recompiles what has changed. The output executable is placed in a new target/debug/ directory.
  2. Run: It then immediately executes the compiled program.

You should see the following output in your terminal:

   Compiling simplilang_compiler v0.1.0 (/path/to/your/simplilang_compiler)
    Finished dev [unoptimized + debuginfo] target(s) in 0.50s
     Running `target/debug/simplilang_compiler`
Hello, world!

Success! You now have a valid, runnable Rust project. This simple “Hello, world!” program serves as our blank canvas. We will soon replace it with the powerful logic of our compiler.

Next Steps

We’ve successfully created the skeleton of our Rust project. The next crucial step is to tell Cargo about the external libraries we’ll rely on. We’ll edit the Cargo.toml file to add inkwell—the Rust wrapper for the LLVM API—which will be the bridge between our Rust code and the LLVM toolchain you installed in the previous step.

Further Reading

Add inkwell, clap, and a parser-combinator crate like chumsky or nom to your Cargo.toml.

Mục tiêu:

Having successfully created the skeleton for our simplilang_compiler project, we now need to equip it with the necessary tools for the job. In the Rust ecosystem, external libraries are called crates, and we manage them in the Cargo.toml manifest file you saw in the previous step. This file acts as the blueprint for our project, telling the Cargo build tool everything it needs to know, including which external crates to download and link.

For our compiler, we need three key libraries to start:

  1. A way to talk to LLVM.
  2. A way to build a professional command-line interface (CLI).
  3. A way to parse our SimpliLang source code.

Let’s add these to our project.

The Three Core Dependencies

Open your Cargo.toml file. You’ll see an empty [dependencies] section at the bottom. This is where we will add our crates.

1. inkwell: The Bridge to LLVM

This is arguably the most important dependency for our project. inkwell is a high-level, safe Rust wrapper for the LLVM-C API. It allows us to interact with the powerful LLVM toolchain you installed in the first task directly from our Rust code. We’ll use it to construct LLVM’s Intermediate Representation (IR) without having to write unsafe C++ or C code.

A critical detail about inkwell is that its build script needs to know which version of the LLVM library to link against on your system. We do this using feature flags. Since we installed LLVM 15, we must enable the corresponding feature flag in inkwell.

2. clap: The Command-Line Argument Parser

Every good compiler needs a solid command-line interface. Users will need to pass it a source file, specify an output file, and control options like optimizations. clap (Command Line Argument Parser) is the de-facto standard for building powerful, fast, and user-friendly CLIs in Rust.

We will use clap’s “derive” feature, which allows us to define our entire CLI structure using a simple Rust struct and attributes. This makes parsing arguments incredibly easy and robust.

3. chumsky: The Parser-Combinator Framework

Before we can generate code, we need to understand the structure of the SimpliLang program the user has written. This process is called parsing, and it involves turning a stream of tokens (like let, x, =, 10, ;) into a structured representation called an Abstract Syntax Tree (AST).

While you could write a parser from scratch, this is often complex and error-prone. A much more productive and robust approach is to use a parser-combinator library. These libraries provide a set of building blocks (the “combinators”) that you can use to compose complex parsers from simpler ones, mirroring the structure of your language’s grammar.

For this project, we’ll use chumsky. It’s a modern parser-combinator crate that is particularly well-suited for compilers because it has excellent support for generating human-friendly, detailed error messages, which is invaluable for the end-users of our language.

Updating Cargo.toml

Now, let’s add these three dependencies to the [dependencies] section of your Cargo.toml file. Replace the empty section with the following:

[dependencies]
inkwell = { version = "0.1.0-beta.5", features = ["llvm15-0"] }
clap = { version = "4.4.18", features = ["derive"] }
chumsky = "0.9.3"

Let’s break down this configuration:

  • inkwell = { version = "0.1.0-beta.5", features = ["llvm15-0"] }: We specify the exact version of inkwell and, most importantly, enable the llvm15-0 feature. This tells inkwell’s build script to look for the LLVM 15 libraries you installed. If you miss this step, the build will fail!
  • clap = { version = "4.4.18", features = ["derive"] }: We specify the version of clap and enable the derive feature, which lets us define our CLI declaratively.
  • chumsky = "0.9.3": For chumsky, we just specify a version. Cargo uses semantic versioning rules, so this will get version 0.9.3 or any later compatible patch release.

Verifying the Setup

After you’ve saved your Cargo.toml file with these changes, it’s time to let Cargo work its magic. Go to your terminal, make sure you are in the simplilang_compiler directory, and run the following command:

cargo check

This command does several important things:

  1. Resolves Dependencies: Cargo reads your Cargo.toml, looks up the dependencies on crates.io (the official Rust package registry), and resolves the exact versions for all crates and their sub-dependencies.
  2. Creates Cargo.lock: It will create a new file named Cargo.lock. This file is a snapshot of the exact versions of all dependencies used. Its purpose is to ensure that every time you (or anyone else) build this project, you get the exact same versions, leading to reproducible builds. You should commit this file to your version control.
  3. Downloads and Compiles: Cargo downloads the source code for inkwell, clap, chumsky, and all of their dependencies. It then compiles them. This will take a few minutes the first time, especially for a large C++-dependent crate like inkwell.
  4. Checks Your Code: Finally, it checks your own project’s code (src/main.rs) for any errors without producing a final executable. This is faster than a full cargo build.

If the command finishes without any errors, you’ll see something like this:

    Updating crates.io index
   Compiling proc-macro2 v1.0.78
   Compiling unicode-ident v1.0.12
   ... (many, many more lines of compilation) ...
   Compiling llvm-sys v150.1.1
   Compiling inkwell v0.1.0-beta.5
   Compiling simplilang_compiler v0.1.0 (/path/to/your/simplilang_compiler)
    Finished dev [unoptimized + debuginfo] target(s) in 2m 15s

Success! Your project is now configured with its core dependencies. The bridge to LLVM is in place, and the tools for parsing our language and its command-line arguments are ready to be used.

Next Steps

With our dependencies added and compiled, we are now ready to write our first piece of code that actually interacts with LLVM. In the next task, we will modify src/main.rs to use inkwell to initialize an LLVM context and create a basic module.

Further Reading

Initialize LLVM Context and Module in Rust

Mục tiêu: Learn about the foundational LLVM Context and Module structures and write a Rust program using the inkwell crate to initialize them, verifying the connection to the LLVM toolchain.

Excellent, you’ve successfully configured your project with the necessary dependencies. With the inkwell crate now part of our project, we have a direct line of communication to the powerful LLVM toolchain. It’s time to write our first lines of code to verify this connection. We will replace the default “Hello, world!” program with a small test program that initializes the core components needed for any compilation task using LLVM.

The Two Foundational LLVM Structures: Context and Module

Before we write the code, it’s crucial to understand the two most fundamental concepts we’ll be interacting with in LLVM: the Context and the Module.

The LLVM Context

Think of the Context as the master container for a single compilation session. It’s an opaque object that owns and manages the memory for all of the core LLVM data structures. Every significant object you create—like types, functions, and basic blocks—will be associated with a specific Context. You can’t mix objects from different contexts. For our compiler, which will compile one file at a time, we will create a single, top-level Context at the beginning of our program and use it for the entire duration of the compilation.

In inkwell, this is represented by the inkwell::context::Context struct. It’s the entry point to the entire LLVM API.

The LLVM Module

If the Context is the entire universe, a Module is a single galaxy within it. A Module is LLVM’s top-level container for a single “translation unit” of code. In simpler terms, it’s the LLVM equivalent of a single source file (like a .c file in C or a .rs file in Rust).

Each Module will contain:

  • The functions defined in the source code.
  • Any global variables.
  • Type definitions.
  • Metadata about the target architecture (e.g., “x86_64-pc-linux-gnu”).

Our compiler’s goal for a given SimpliLang file will be to construct a complete LLVM Module that represents the entire program. We will then pass this Module to the LLVM backend to be optimized and compiled into machine code.

In inkwell, this is represented by the inkwell::module::Module struct. You always create a Module from a Context.

Initializing LLVM in main.rs

Now, let’s put these concepts into practice. Open your src/main.rs file and replace its entire contents with the following code. This program will initialize a Context and then use it to create our first, albeit empty, Module.

// src/main.rs

// We need to bring the Context type into scope to use it.
// This is the main entry point to the inkwell API.
use inkwell::context::Context;

fn main() {
    // In Rust, `let` is used to declare a variable.
    // Here, we create the top-level LLVM `Context`.
    // The `Context::create()` function is a static method on the Context struct
    // that initializes a new context instance. This context will manage the memory
    // for all our LLVM objects.
    let context = Context::create();

    // Now, we use our `context` to create a `Module`.
    // A module in LLVM is like a single source file in C. It's a container
    // for functions, global variables, etc.
    // We give our module a name, "simplilang_main", which is useful for debugging
    // and identification in LLVM IR.
    let module = context.create_module("simplilang_main");

    // The Builder is an essential tool for constructing LLVM IR. It provides
    // a structured and convenient API for creating instructions and inserting
    // them into basic blocks. We will use it extensively in later steps.
    // We create it here to show its relationship with the context.
    let _builder = context.create_builder();

    // The ExecutionEngine is what allows us to JIT (Just-In-Time) compile
    // our module and run it directly from our program. While we will focus
    // on AOT (Ahead-Of-Time) compilation to an executable file, understanding
    // that an ExecutionEngine exists is useful. We create it but don't use it yet.
    // The `expect` call will cause the program to panic if the engine can't be created.
    let _execution_engine = module.create_jit_execution_engine(inkwell::OptimizationLevel::None)
        .expect("Failed to create Execution Engine");

    println!("✅ LLVM Context and Module initialized successfully!");
}

Code Breakdown

Let’s dissect this code line-by-line:

  1. use inkwell::context::Context;: The use keyword in Rust brings a type, trait, or function into the current scope, so we can refer to it directly. Here, we’re importing the Context struct from the inkwell crate. Without this, we would have to write the full path inkwell::context::Context::create().
  2. let context = Context::create();: This is the most important first step. We call the static method create() on the Context struct. This initializes the underlying LLVM context and returns a safe Rust wrapper around it. The variable context now holds our compilation session’s state.
  3. let module = context.create_module("simplilang_main");: Here, we use an instance method on our context object to create a new Module. The method takes a string slice (&str) as the module’s name. This name will appear at the top of the generated LLVM IR file, which is very helpful for debugging.

  4. let _builder = context.create_builder(); and let _execution_engine = ...: We also initialize a Builder and an ExecutionEngine. While we won’t use them in this specific task, it’s good practice to introduce them now as they are core components we will rely on heavily.

    • A Builder is a helper object that makes it easy to create LLVM instructions and append them to a code block.
    • An ExecutionEngine is what can take an in-memory LLVM Module and execute it directly, a process known as JIT compilation.
    • The underscore _ prefix on the variable names (_builder, _execution_engine) is a Rust convention to tell the compiler, “I know this variable is unused, please don’t show me a warning.”
  5. println!(...);: Finally, we print a success message to the console. If our program runs and prints this message without crashing (or “panicking” in Rust terminology), it means we have successfully linked against the LLVM libraries and initialized the core inkwell structures.

Running Your Program

Now, head back to your terminal, ensure you are in the simplilang_compiler directory, and run the program:

cargo run

You should see output similar to this:

   Compiling simplilang_compiler v0.1.0 (/path/to/your/simplilang_compiler)
    Finished dev [unoptimized + debuginfo] target(s) in 0.83s
     Running `target/debug/simplilang_compiler`
✅ LLVM Context and Module initialized successfully!

Congratulations! This simple output is a huge milestone. It confirms that your Rust project can successfully communicate with the LLVM C++ libraries you installed earlier. The bridge is built.

Next Steps

We’ve created an empty canvas—our Module. The next logical step is to start painting on it. We will add the most fundamental part of any executable program: a main function. In the next task, you will learn how to define a function within the LLVM module and make it return a simple integer constant.

Further Reading

Define a main Function in LLVM

Mục tiêu: Extend an LLVM module by defining a main function that takes no arguments and returns a constant integer value of 0. This involves using the inkwell builder in Rust to create the function signature, add a basic block, and generate the return instruction.

In the previous task, we successfully initialized the core LLVM Context and an empty Module. This was a critical first step, proving that our Rust program can communicate with the LLVM library. Now, we’ll take that empty module and add the most fundamental piece of any executable program: the main function. Our goal is to define a main function that does the simplest possible thing: return the integer constant 0.

Core Concepts: Functions, Types, and Basic Blocks

To generate code with LLVM, we need to understand a few key concepts that form the structure of its Intermediate Representation (IR).

LLVM Types

Just like in high-level languages, every value in LLVM has a type. Before we can define a function, we must first define its signature, which includes its return type and the types of its parameters. Since the standard C main function returns an integer, we’ll need an integer type. In inkwell, we can get standard types like a 32-bit integer directly from the Context.

LLVM Functions

A function in LLVM (FunctionValue in inkwell) is a container for executable code within a Module. Each function has a type, a name, and a body. We will add a function named "main" to our module with the type i32 (), which means “a function that takes no arguments and returns a 32-bit integer.”

Basic Blocks

The body of an LLVM function isn’t just a flat list of instructions. It’s composed of one or more Basic Blocks (BasicBlock in inkwell). A basic block is a sequence of instructions that are always executed in order. It has one entry point (the first instruction) and must end with a special “terminator” instruction, which passes control flow elsewhere (like returning from the function or branching to another block). Every function must have at least one basic block, conventionally called the “entry” block.

The IR Builder

In the last step, we created a Builder but didn’t use it. Now, its role becomes central. The Builder is a helper object that makes it easy to create LLVM instructions. Instead of manually creating an instruction and inserting it into a block, we “position” the builder at the location where we want to add code (e.g., at the end of our entry block) and then use its methods like build_return to generate and insert the instruction for us.

Building the main Function

Let’s modify our src/main.rs to put these concepts into action. We will build upon our previous code, adding the necessary logic to define our main function.

Here is the updated code for src/main.rs. The changes are highlighted with detailed comments.

// src/main.rs

// We need to bring more types into scope from inkwell
use inkwell::context::Context;

fn main() {
    // === Set up the basics ===
    let context = Context::create();
    let module = context.create_module("simplilang_main");
    // We will now use the builder, so we remove the `_` prefix
    let builder = context.create_builder();

    // === Define the function type ===
    // First, we get the 32-bit integer type from the context.
    // This will be the return type of our 'main' function.
    let i32_type = context.i32_type();

    // Next, we define the function's signature.
    // `fn_type` takes an array of parameter types and a boolean for variadic arguments.
    // Our main function will have no parameters (`&[]`) and is not variadic (`false`).
    let main_fn_type = i32_type.fn_type(&[], false);

    // === Add the function to the module ===
    // Now we can add a function with our desired signature to the module.
    // We give it the name "main". The `None` specifies default linkage.
    let main_fn = module.add_function("main", main_fn_type, None);

    // === Create the function's body ===
    // A function's body is made up of basic blocks.
    // We create a basic block named "entry" in our main function.
    let entry_block = context.append_basic_block(main_fn, "entry");

    // We now position the builder at the end of our new "entry" block.
    // Any instructions we generate from now on will be placed here.
    builder.position_at_end(entry_block);

    // === Create the return instruction ===
    // We need a value to return. Let's create a constant 32-bit integer with value 0.
    // The `false` indicates that the value is not signed.
    let i32_zero = i32_type.const_int(0, false);

    // Now, we use the builder to create the return instruction.
    // `build_return` takes an optional reference to the value to return.
    // Since our function is supposed to return an i32, we provide our zero constant.
    builder.build_return(Some(&i32_zero));

    // We can update our success message to reflect the new progress.
    println!("✅ `main` function defined successfully!");
}

Dissecting the New Code

  1. Removing the Underscore: We’ve changed let _builder to let builder. This signals our intent to actually use this variable, so the Rust compiler won’t warn us about it being unused.
  2. Getting the Type: let i32_type = context.i32_type(); is our first interaction with LLVM’s type system. We ask the context for a handle to its representation of a 32-bit integer.
  3. Defining the Signature: i32_type.fn_type(&[], false) creates a function type. The first argument is a slice of parameter types (empty in our case), and the second indicates if the function is variadic (like C’s printf).
  4. Creating the Function: module.add_function(...) officially adds a FunctionValue to our module. This is the container for the function’s code.
  5. Appending a Basic Block: context.append_basic_block(main_fn, "entry") creates the first (and only) basic block for our main function and gives it the label “entry”, which is useful for readability in the generated IR.
  6. Positioning the Builder: builder.position_at_end(entry_block) is a crucial step. It tells the builder, “Get ready to insert instructions right before the end of the entry_block.”
  7. Creating a Constant: i32_type.const_int(0, false) creates an LLVM constant value. This is not a Rust i32; it’s an LLVM IntValue that lives within our Context.
  8. Building the Return: builder.build_return(Some(&i32_zero)) is the climax. We use the builder to generate a ret i32 0 instruction and insert it at the position we set earlier. This “terminates” the basic block and completes our function definition.

If you run cargo run now, the output will still be a simple success message. However, under the hood, we have constructed a complete, valid LLVM module in memory that represents a program that returns 0.

Next Steps

We’ve now programmatically built an LLVM module containing a main function. But how can we be sure it’s correct? The next step is to make our program print the textual representation of this module—the famous LLVM Intermediate Representation (IR)—to the console. This will give us our first look at the code our compiler is generating.

Further Reading

Inspecting the Generated LLVM IR

Mục tiêu: Modify the Rust program to verify the in-memory LLVM module and print its textual Intermediate Representation (IR) to the console for inspection.

You’ve successfully constructed a complete, albeit simple, LLVM module in memory. While your program reports success, this in-memory representation is invisible to us. As a compiler developer, one of your most crucial tools is the ability to inspect the Intermediate Representation (IR) you are generating. This allows you to verify that your compiler’s logic is correctly translating the source language into LLVM’s language.

In this step, we will bridge the gap between the invisible in-memory module and a tangible, human-readable output. We’ll modify our program to print the textual representation of our LLVM module to the console.

What is LLVM Intermediate Representation (IR)?

LLVM IR is the backbone of the entire LLVM ecosystem. It’s a low-level, strongly-typed language with a syntax that resembles an idealized assembly language. It is designed to be the single, universal representation for code within the LLVM system, regardless of the source language (SimpliLang, C++, Rust, Swift) or the target hardware (x86, ARM).

Our compiler’s primary job is to translate SimpliLang code into this IR. Once we have it, LLVM’s powerful backend tools can take over to perform optimizations and generate native machine code.

The inkwell Module struct that we created holds this IR in its in-memory format. Fortunately, it provides a simple method to serialize this format into its textual representation.

Printing the Module to the Console

We will now add a few lines to our src/main.rs to get this textual representation and print it. We will also introduce a best-practice step: verifying the module’s integrity before printing. LLVM provides a verifier that checks for common errors in the generated IR, such as malformed instructions or type mismatches. Catching these errors early can save hours of debugging later.

Let’s update src/main.rs. The new lines are at the very end of the main function.

// src/main.rs

use inkwell::context::Context;

fn main() {
    // === Set up the basics ===
    let context = Context::create();
    let module = context.create_module("simplilang_main");
    let builder = context.create_builder();

    // === Define the function type ===
    let i32_type = context.i32_type();
    let main_fn_type = i32_type.fn_type(&[], false);

    // === Add the function to the module ===
    let main_fn = module.add_function("main", main_fn_type, None);

    // === Create the function's body ===
    let entry_block = context.append_basic_block(main_fn, "entry");
    builder.position_at_end(entry_block);

    // === Create the return instruction ===
    let i32_zero = i32_type.const_int(0, false);
    builder.build_return(Some(&i32_zero));

    // === NEW: Verify and print the IR ===

    // Before printing, it's a good practice to verify the module.
    // The `verify()` method will check for any inconsistencies or errors
    // in the generated IR. If it finds an issue, it will return an `Err`,
    // and `.unwrap()` will cause the program to panic with a helpful message.
    // For a valid module like ours, this will do nothing.
    module.verify().unwrap();

    // Now, we get the textual representation of the module.
    // `print_to_string()` returns an `LLVMString`, which is a special wrapper.
    // We call `.to_string()` on it to convert it into a regular Rust `String`.
    let ir_string = module.print_to_string().to_string();

    // Finally, print the generated IR to the console.
    println!("Generated LLVM IR:\n{}", ir_string);
}

Code Analysis

The core changes are just three new lines:

  1. module.verify().unwrap();: We call the verify method on our module. This runs LLVM’s internal verifier. If the IR is valid, it returns Ok(()). If it’s invalid, it returns an Err containing a description of the problem. We use .unwrap() as a shortcut for this test program; it will cause the program to crash (panic) and print the error if verification fails. In our real compiler, we would handle this error more gracefully.

  2. let ir_string = module.print_to_string().to_string();: This is where the magic happens.

    • module.print_to_string(): This method serializes the entire in-memory module into LLVM’s textual IR format. It returns a special inkwell::support::LLVMString type, which is a wrapper around a C string managed by LLVM to avoid unnecessary memory copies.
    • .to_string(): To use this IR with Rust’s standard I/O functions like println!, we convert the LLVMString into a standard, heap-allocated Rust String.
  3. println!("Generated LLVM IR:\n{}", ir_string);: We use Rust’s standard printing macro to output our generated code to the console, prefixed with a descriptive header.

Expected Output

Now, run your program again from the terminal with cargo run. This time, instead of a simple success message, you will see the actual LLVM IR you have generated:

   Compiling simplilang_compiler v0.1.0 (/path/to/your/simplilang_compiler)
    Finished dev [unoptimized + debuginfo] target(s) in 0.23s
     Running `target/debug/simplilang_compiler`
Generated LLVM IR:
; ModuleID = 'simplilang_main'
source_filename = "simplilang_main"

define i32 @main() {
entry:
  ret i32 0
}

This output is a monumental achievement. It’s the “assembly code” for the LLVM virtual machine, and it perfectly represents the program we intended to build:

  • ; ModuleID = 'simplilang_main': A comment indicating the name we gave our module.
  • source_filename = "simplilang_main": Metadata indicating the source.
  • define i32 @main() { ... }: This defines a function.
    • define: The keyword to define a function.
    • i32: The return type (a 32-bit integer).
    • @main: The function’s name. Global identifiers like function names are prefixed with @.
    • (): The parameter list (empty).
  • entry:: The label for our basic block.
  • ret i32 0: The terminator instruction for the block. It returns a value of type i32 with the constant value 0.

Next Steps

You have now successfully generated valid, human-readable LLVM IR. The next and final task in this setup step is to take this IR and use the LLVM toolchain on your machine to compile it into a native, executable file that you can run directly from your shell.

Further Reading

Compile LLVM IR into a Native Executable

Mục tiêu: Learn how to transform textual LLVM Intermediate Representation (IR) into a runnable native program using the LLVM toolchain. This task covers compiling the IR to an object file with llc and linking it into a final executable with clang, and includes creating a shell script to automate the process.

Brilliant! You’ve successfully generated your first piece of LLVM Intermediate Representation (IR). That text output from the last step is the tangible result of our compiler’s work so far. But what is it good for? An IR string isn’t a program you can run.

The crucial next step is to take this textual IR and transform it into a native, executable program. This process will use the command-line tools from the LLVM toolchain you installed in the very first step, closing the loop and proving that our entire setup is working from end-to-end. This is where the code we’ve written in Rust meets the real world of machine code.

The Two-Stage Path from IR to Executable

Turning LLVM IR into a runnable program is a two-stage process:

  1. Compilation: We use the LLVM Static Compiler (llc) to convert the human-readable LLVM IR (.ll file) into a native object file (.o or .obj file). An object file contains machine code specific to your computer’s architecture (like x86-64 or ARM64), but it’s not a complete program yet. It’s a “relocatable” piece of code that doesn’t know how to start itself or interact with the operating system.
  2. Linking: We use a linker (most easily invoked via a C compiler like clang or gcc) to take our object file and “link” it with the necessary system libraries. The linker resolves all the final details, bundles in the C standard library’s startup code (which is responsible for setting up the environment and calling your main function), and produces the final, standalone executable file.

Let’s walk through this process step-by-step using manual commands.

Step 1: Capturing the LLVM IR

First, we need to save the IR generated by our Rust program into a file. The easiest way to do this is with shell redirection.

Open your terminal in the simplilang_compiler project directory and run:

cargo run > output.ll

This command does two things:

  • cargo run: Compiles and runs our Rust program as before.
  • >: This is the shell’s “redirect output” operator. Instead of printing the LLVM IR to the console, it captures all of the program’s standard output and writes it into a new file named output.ll.

After the command finishes, you will have a new file, output.ll, in your project directory. You can verify its contents using cat output.ll (on Linux/macOS) or type output.ll (on Windows). It should contain the exact IR you saw previously.

Step 2: Compiling the IR to an Object File with llc

Now that we have our IR in a file, we can use llc, the LLVM compiler. This tool reads the .ll file and emits an object file.

Run the following command in your terminal. Remember to use the versioned command (llc-15, clang-15) if that’s what you installed.

# On macOS/Linux
llc-15 -filetype=obj output.ll -o output.o

# On Windows (the command might just be 'llc')
llc -filetype=obj output.ll -o output.o

Let’s break down this command:

  • llc-15: The LLVM static compiler.
  • -filetype=obj: This flag is crucial. It tells llc that we want an object file as the output. Without it, llc would default to producing an assembly file (.s).
  • output.ll: This is the input file containing our LLVM IR.
  • -o output.o: The -o flag specifies the name for the output file. We’re naming it output.o, which is the standard convention for object files on Linux and macOS. On Windows, this would typically be output.obj.

After this command, you’ll have a new file, output.o. This is a binary file containing pure machine code. You can’t run it directly, but it’s the raw compiled version of our main function.

Step 3: Linking the Object File into an Executable with clang

Finally, we link our object file to create the final executable. We’ll use clang for this, as it knows how to find the right system libraries and startup code.

# On macOS/Linux
clang-15 output.o -o main_program

# On Windows (the command might just be 'clang')
clang output.o -o main_program.exe

Here’s the breakdown:

  • clang-15: The C language frontend for LLVM, which we’ll use as our linker driver.
  • output.o: The input object file we just created.
  • -o main_program: The -o flag again specifies the output file name. This time, we’re creating our final executable. We’ve named it main_program (on Windows, it’s conventional to add the .exe extension).

Congratulations! You should now have a runnable executable file in your directory. You have successfully completed the entire compilation pipeline manually.

Creating a Reusable Script

Typing these commands every time can be tedious. Let’s create a simple shell script to automate this process.

Create a new file named compile.sh in your project root.

For Linux and macOS: 

#!/bin/bash

# This script compiles the LLVM IR generated by our Rust program
# into a final executable.

# Stop the script if any command fails
set -e

# 1. Generate the LLVM IR from our Rust compiler
echo "=== Generating LLVM IR... ==="
cargo run > output.ll

# 2. Compile the LLVM IR into an object file
echo "=== Compiling IR to object file... ==="
llc-15 -filetype=obj output.ll -o output.o

# 3. Link the object file into an executable
echo "=== Linking object file into an executable... ==="
clang-15 output.o -o main_program

# 4. Clean up intermediate files
rm output.ll output.o

echo "=== Done! Executable created: ./main_program ==="

Save the file. Before you can run it, you need to make it executable:

chmod +x compile.sh

Now, you can compile your entire project with a single command:

./compile.sh

This script automates all the manual steps, including cleaning up the intermediate .ll and .o files, leaving you with just the final main_program executable.

Next Steps

We have successfully generated LLVM IR and compiled it into a native executable. The very next task is to run this program and verify that it behaves as expected—specifically, that it returns an exit code of 0. This final check will provide absolute confirmation that our end-to-end toolchain is fully functional.

Further Reading

Run Executable and Check Exit Code

Mục tiêu: Execute the compiled native program and verify its successful execution by checking for an exit code of 0 using shell-specific commands for Linux, macOS, and Windows.

You have successfully navigated the entire compilation toolchain, starting from Rust code and ending with a native executable file named main_program (or main_program.exe). This is the moment of truth. Does the program actually work? The final piece of the puzzle is to run this executable and verify that it behaves exactly as we designed it to: exit with a success code of 0.

Understanding Program Exit Codes

When you run a command-line program, it performs its tasks and then exits. Upon exiting, it returns a small integer number to the operating system, known as an exit code or status code. This code is the program’s final message to the world, indicating whether it completed successfully or encountered an error.

The universal convention is: * An exit code of 0 means success. The program ran without any issues. * Any non-zero exit code (typically 1-255) means failure. The specific number can be used to signify different types of errors.

Our generated LLVM IR contained the instruction ret i32 0 inside the main function. When compiled and linked, this instruction is what tells the operating system’s runtime that our program should terminate with an exit code of 0.

Running our program won’t produce any visible output like “Hello, world!” because we never instructed it to print anything. Its success is silent and communicated solely through its exit code. Our job now is to run the program and then ask the shell what that exit code was.

Executing the Program and Checking its Status

Let’s run the executable you created in the last step. The process is slightly different depending on your operating system’s shell.

On Linux and macOS

  1. Run the Executable: In your terminal, from the simplilang_compiler directory, run the following command:

    bash ./main_program * Why ./? The . represents the current directory. By default, for security reasons, most shells don’t look for executables in the current directory. The ./ prefix explicitly tells the shell: “Find and run the file named main_program located right here.” After you press Enter, it will seem like nothing happened. The program will run instantly, return 0, and your shell prompt will reappear. This is the expected behavior.

  2. Check the Exit Code: Immediately after the program finishes, run this command:

    bash echo $? * What is $?? This is a special variable in POSIX-compliant shells (like bash and zsh) that automatically stores the exit code of the most recently executed foreground command. The echo command simply prints the value of this variable to the console.

You should see the following output:

0

On Windows (using Command Prompt - cmd.exe)

  1. Run the Executable:

    bash .\main_program.exe

  2. Check the Exit Code:

    bash echo %ERRORLEVEL% * What is %ERRORLEVEL%? This is the Command Prompt’s equivalent of $?. It holds the exit code of the last command that was run.

You should see the following output:

0

On Windows (using PowerShell)

  1. Run the Executable:

    bash .\main_program.exe

  2. Check the Exit Code:

    bash $LASTEXITCODE * What is $LASTEXITCODE? PowerShell uses this automatic variable to store the exit code of the last Win32 application that was run.

Again, the output should be 0.

A Momentous Occasion!

Seeing that 0 is a huge milestone. It’s the final confirmation that every single piece of our toolchain is working in perfect harmony:

  1. Your Rust program successfully used the inkwell crate to create a valid LLVM module in memory.
  2. The inkwell library correctly serialized this module into the textual LLVM IR format.
  3. The llc compiler correctly parsed the IR and compiled it into native machine code for your computer’s architecture.
  4. The clang linker correctly took that machine code, bundled it with the necessary system startup routines, and produced a valid, runnable executable.
  5. Your operating system was able to load and run this executable, which correctly returned the success code of 0.

You have successfully built and run your first program compiled from your own compiler’s frontend!

Next Steps

The foundation is now rock-solid. We’ve proven the end-to-end LLVM integration works. Before we start diving into the complexities of parsing SimpliLang, our final setup task is to organize our project. We will create a clean directory structure within src/ to house the future components of our compiler: the lexer, parser, semantic analyzer, and code generator.

Further Reading

Organize Rust Compiler Project Structure

Mục tiêu: Create a clean and scalable project structure for the SimpliLang compiler by setting up Rust modules for core components like the lexer, parser, AST, and code generator.

Of course, let’s get your project organized!

Finalizing the Foundation: A Clean Project Structure

Congratulations! You have successfully navigated the entire compilation pipeline, from generating LLVM IR in Rust to running a native executable. This confirms that your development environment is perfectly configured. This is a significant milestone.

Before we dive into the exciting work of designing SimpliLang and writing the compiler’s logic, we have one final, crucial setup task. Right now, all our test code lives in a single file: src/main.rs. For a small test, this is fine. For a real compiler with distinct phases—lexing, parsing, semantic analysis, and code generation—this will quickly become unmanageable.

The final step in laying our foundation is to create a clean, scalable directory structure. This practice, known as separation of concerns, is a cornerstone of professional software engineering. It makes your code easier to navigate, understand, test, and maintain.

The Rust Module System: From Files to Features

In Rust, the project’s directory structure is deeply integrated with its powerful module system. A module is a namespace that contains definitions of functions, structs, enums, traits, and even other modules. This is how Rust allows you to split your code across multiple files and directories.

There are two primary ways to declare a module:

  1. In a file: If you have a file named src/parser.rs, you can declare it as a module in src/main.rs by writing mod parser;.
  2. In a directory: If you want a module that might contain its own sub-modules, you create a directory (e.g., src/parser/) and place a special file named mod.rs inside it. This mod.rs file becomes the root of the parser module.

For our compiler, we will use the second approach, as it gives us the most flexibility for future expansion.

Creating the Compiler’s Directory Structure

Let’s create the directories that will house the core components of our compiler.

  1. ast: This module will define the data structures for our Abstract Syntax Tree. The AST is the tree-like representation of the source code’s structure that the parser produces.
  2. lexer: This module will contain the lexer (or tokenizer), which is responsible for converting the raw source code string into a stream of tokens.
  3. parser: This module will house the parser, which takes the token stream from the lexer and builds the AST.
  4. codegen: This module will be responsible for code generation. It will traverse the AST and generate the corresponding LLVM IR.

Open your terminal in the simplilang_compiler project root and run the following commands to create this structure:

# Create the main component directories inside src/
mkdir src/ast src/lexer src/parser src/codegen

# Create the root module file for each new directory
touch src/ast/mod.rs src/lexer/mod.rs src/parser/mod.rs src/codegen/mod.rs

After running these commands, your src/ directory should look like this:

src/
├── ast/
│   └── mod.rs
├── codegen/
│   └── mod.rs
├── lexer/
│   └── mod.rs
├── main.rs
└── parser/
    └── mod.rs

Declaring the Modules in the Crate Root

Now that the files and directories exist, we need to tell the Rust compiler about them. We do this by declaring them in our crate root, which for a binary project is src/main.rs.

Open src/main.rs and add the module declarations to the top of the file. For now, we will leave the existing test code in the main function. We’ll move it to the codegen module where it belongs in a later step.

Here is the updated src/main.rs. The only change is adding the four new lines at the top.

// src/main.rs

// NEW: Declare the modules we just created.
// This makes the code inside `ast/mod.rs`, `lexer/mod.rs`, etc.,
// available to our crate.
pub mod ast;
pub mod codegen;
pub mod lexer;
pub mod parser;

use inkwell::context::Context;

fn main() {
    // === Set up the basics ===
    // (The rest of your existing test code remains here for now)
    let context = Context::create();
    let module = context.create_module("simplilang_main");
    let builder = context.create_builder();

    // === Define the function type ===
    let i32_type = context.i32_type();
    let main_fn_type = i32_type.fn_type(&[], false);

    // === Add the function to the module ===
    let main_fn = module.add_function("main", main_fn_type, None);

    // === Create the function's body ===
    let entry_block = context.append_basic_block(main_fn, "entry");
    builder.position_at_end(entry_block);

    // === Create the return instruction ===
    let i32_zero = i32_type.const_int(0, false);
    builder.build_return(Some(&i32_zero));

    // === Verify and print the IR ===
    module.verify().unwrap();
    let ir_string = module.print_to_string().to_string();
    println!("Generated LLVM IR:\n{}", ir_string);
}

By adding pub mod ast; (and the others), we are:

  • Declaring the module exists.
  • Making it pub (public), which means other modules in our crate can access its contents (once we add some).

Verifying the New Structure

To confirm that Rust understands our new project structure, run the cargo check command. This will analyze your code and check for errors without doing a full compilation.

cargo check

You should see it complete successfully:

    Checking simplilang_compiler v0.1.0 (/path/to/your/simplilang_compiler)
    Finished dev [unoptimized + debuginfo] target(s) in 0.08s

Success! Even though our new module files are empty, Rust recognizes the structure and confirms that the project is well-formed.

Next Steps

This task concludes the entire first step of our project. We have successfully set up the LLVM toolchain, created a Rust project, integrated it with LLVM, verified the end-to-end compilation process, and organized our project for future development. The foundation is now complete and solid.

Now, the real fun begins. We will move on to Step 2: Designing the Syntax and Semantics of SimpliLang. In the very next task, we will start defining the rules of our language, beginning with its primitive data types.

Further Reading