Create a New Rust Project with Cargo

Mục tiêu: Learn how to use the cargo new command to create the initial project structure for a Rust application. This task covers the basic project scaffolding for building a concurrent web scraper.

Welcome to the beginning of your journey in building a Concurrent Web Scraper with Rust! Every great project starts with a solid foundation, and in the Rust ecosystem, that foundation is built with Cargo.

Understanding Cargo: Your Rust Project Manager

Before we type our first command, let’s briefly discuss the tool we’re about to use. Cargo is Rust’s official build system and package manager. Think of it as your indispensable assistant for any Rust project. It handles a multitude of tasks, including:

Project Scaffolding: Creating the standard directory structure and configuration files for new projects.
Dependency Management: Fetching and compiling external libraries (called “crates” in Rust) that your project depends on.
Building Your Code: Compiling your Rust code into an executable binary or a library.
Running Tests: Executing the test suites you write for your code.
And much more!

By using Cargo from the start, we ensure our project follows standard conventions, making it easier to manage, share, and for others to contribute to.

Creating Your Project

Our first task is to create a new binary application project. A “binary” project is one that compiles into an executable file that you can run directly from your terminal. We will use the cargo new command to accomplish this. This command instructs Cargo to create a new directory with all the necessary boilerplate to get started.

Open your terminal or command prompt and run the following command:

cargo new concurrent_scraper

When you execute this, Cargo will perform a few actions:

It creates a new directory named concurrent_scraper.
Inside this directory, it initializes a new Git repository, which is a best practice for version control.
It generates a standard project structure, which we will look at next.

You will see a confirmation message in your terminal:

Created binary (application) `concurrent_scraper` package

Exploring the Project Structure

Let’s examine the files and directories that Cargo just created for us. If you list the contents of the concurrent_scraper directory, you’ll see the following structure:

concurrent_scraper/
├── .git/
├── .gitignore
├── Cargo.toml
└── src/
    └── main.rs

Let’s break down what each of these is for:

Cargo.toml: This is the manifest file for your project. The .toml extension stands for “Tom’s Obvious, Minimal Language”. This file contains all the metadata for your project, such as its name, version, author, and—most importantly for the upcoming tasks—its dependencies (the external crates we need). We will be editing this file very soon to add libraries for asynchronous operations, HTTP requests, and more.
src/main.rs: Cargo creates a src directory to hold all of your source code. The default entry point for a binary application is the main.rs file. If you open it, you’ll find a simple “Hello, world!” program:

rust fn main() { println!("Hello, world!"); } This is the heart of your application, and we will be replacing this code with our web scraping logic as we progress.
.gitignore: Since Cargo initializes a Git repository, it also provides a .gitignore file. This file is pre-configured to ignore common files and directories that shouldn’t be committed to version control, like the target directory where compilation artifacts are stored.

You have now successfully set up the skeleton of your Rust application. The next task will be to navigate into this newly created project directory and begin configuring our project’s dependencies.

Navigate Into the Project Directory

Mục tiêu: Learn how to use the ‘cd’ command to change your current working directory and enter the newly created Rust project folder. This is a fundamental step for interacting with project files and using Cargo commands.

Excellent! In the previous task, you used cargo new to lay the groundwork for our web scraper. This command created a dedicated directory named concurrent_scraper containing all the essential starting files. Now, to begin working on our project, we first need to enter that directory.

Understanding the Working Directory

Every time you use your terminal or command prompt, you are “located” in a specific directory. This is known as the current working directory. When you run commands, they operate relative to this location. For instance, Cargo commands like cargo build or cargo run need to be executed from within a project’s root directory so they can find the Cargo.toml manifest file and the src folder.

Right now, your terminal is likely in the directory where you ran the cargo new concurrent_scraper command (for example, your Desktop or Projects folder). To interact with our new project’s files and use Cargo’s project-specific features, we must change our current working directory to the concurrent_scraper directory itself.

Changing Your Directory

The command to navigate through your file system is cd, which is short for “change directory”. This is a fundamental and universal command available in virtually every command-line shell, including Bash, Zsh, PowerShell, and the Windows Command Prompt.

To move into your new project’s directory, execute the following command in your terminal:

cd concurrent_scraper

After running this command, you won’t see any dramatic output. The most common feedback is that your terminal’s prompt will update to show that you are now inside the concurrent_scraper directory. It might look something like this (the exact appearance depends on your system and shell configuration):

# Before
~/Projects$

# After
~/Projects/concurrent_scraper$

You are now “inside” the project. Any command you run will be executed from this context. You can verify this by listing the contents of the directory (ls on macOS/Linux or dir on Windows), and you should see the Cargo.toml file and the src directory we discussed in the previous step.

With this simple but crucial step completed, we are now perfectly positioned to start configuring our project’s dependencies in the Cargo.toml file, which is our very next task.

Explore the `Cargo.toml` Manifest File

Mục tiêu: Learn about the Cargo.toml manifest file, the central configuration for a Rust project. Open the file in your editor to understand its structure, including the [package] and [dependencies] sections.

Having successfully navigated into your concurrent_scraper project directory, you are now in the command center of your new Rust application. The next logical step is to familiarize yourself with and open the project’s most important configuration file: Cargo.toml.

The Heart of Your Project: `Cargo.toml`

Every Cargo-based Rust project has a Cargo.toml file at its root. This file is known as the manifest, and it serves as the central configuration for your entire project. Written in the TOML (Tom’s Obvious, Minimal Language) format, it’s designed to be easy for humans to read and write while being straightforward for machines to parse.

Think of Cargo.toml as the blueprint for your application. It tells Cargo everything it needs to know, including: * Metadata: The name of your project, its version, the authors, and the specific Rust edition it targets. * Dependencies: A list of all the external libraries (called crates) that your project needs to function. This is the section we will be focusing on heavily in the upcoming tasks. * Profiles: Configuration for different build types, like development (dev) and release (release), allowing you to specify settings like optimization levels.

Opening the Manifest File

To view and edit this file, you can use any text editor or Integrated Development Environment (IDE) you prefer. A common and highly recommended practice is to open the entire project folder in your editor. This gives you a complete view of all your files and makes navigation much easier.

If you have Visual Studio Code installed and its command-line tool set up, you can open the entire project from your terminal with a single command (remember, you should still be inside the concurrent_scraper directory):

code .

The . is a shorthand that represents the current directory. This command will launch VS Code with your project loaded, and you can then easily find and click on Cargo.toml in the file explorer sidebar. If you use a different editor like Sublime Text or Zed, you can use a similar command (e.g., zed .) or simply open the folder through the editor’s graphical interface.

What’s Inside?

When you open Cargo.toml, you will see the initial content that cargo new generated for you. It will look like this:

[package]
name = "concurrent_scraper"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

Let’s break down what this means:

[package]: This is a TOML “table” or section header. Everything under it defines the metadata for your package (or crate).
- name: The name of your crate. Cargo uses this for naming the output binary and when publishing to crates.io, the official Rust package registry.
- version: The current version of your crate, following the Semantic Versioning standard.
- edition: Specifies the Rust edition your code is written against. An edition is a way for Rust to evolve its syntax and features without breaking older code. 2021 is the latest edition and is the default for new projects.
[dependencies]: This is the section where we will list all the external crates our web scraper depends on. It’s empty right now, but in the very next tasks, we will populate this section with powerful libraries for asynchronous programming (tokio), making web requests (reqwest), parsing HTML (scraper), and more.

You have now opened the control panel for your project. In the following tasks, you will edit this file to pull in the tools required to build our concurrent web scraper.

Add Tokio Dependency to Cargo.toml

Mục tiêu: Add the Tokio asynchronous runtime as a dependency to the project’s Cargo.toml file, enabling the ‘full’ feature to support concurrent operations for a web scraper.

With your Cargo.toml file open, you are ready to add the first and most foundational dependency for our project. We’re building a concurrent web scraper, and to achieve concurrency in modern Rust, we need an asynchronous runtime. For this, we will use Tokio, the industry standard for writing fast and reliable asynchronous applications in Rust.

Why Asynchronous? And What is a Runtime?

Before adding the code, let’s understand why we need this. A traditional (synchronous) scraper would work like this:

Send a request to URL 1.
Wait for the server to respond.
Process the response.
Send a request to URL 2.
Wait for the server to respond.
…and so on.

The “wait” step is the bottleneck. While our program is waiting for a response from a server over the network, the CPU is sitting idle, doing nothing productive. This is incredibly inefficient.

Asynchronous programming allows us to start a task (like a network request) and then move on to other work without waiting for it to complete. When the task is finished, the runtime will notify our program so it can process the result. This means we can send out requests to hundreds of URLs at the same time and process their responses as they arrive. This is the key to building a high-performance scraper.

However, Rust’s async/await syntax itself doesn’t do anything. It’s just a way to define asynchronous logic. We need an engine to actually run these asynchronous tasks, schedule them, and handle the I/O (like network requests). This engine is called a runtime, and Tokio is the most popular and powerful async runtime in the Rust ecosystem.

Adding Tokio with the “full” Feature

To add Tokio to our project, we will add a line to the [dependencies] section of your Cargo.toml file. We will also specify a “feature flag”.

What are Feature Flags? Crates can be large and have a lot of optional functionality. Feature flags allow us to tell Cargo that we only want to compile and include specific parts of a crate. This helps reduce compile times and the size of our final executable.

Tokio has many features, such as networking, timers, and file system utilities. For our project, and for ease of getting started, we will use the full feature. This is a convenient shorthand that enables all of Tokio’s features, ensuring we have everything we need for this project and for future experimentation.

Modify your Cargo.toml file to look like this. The only change is the addition of the tokio line under [dependencies]:

[package]
name = "concurrent_scraper"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "1", features = ["full"] }

Let’s break down this new line: * tokio: This is the name of the crate we want to use. * { ... }: We use a TOML inline table to provide more configuration than just a version number. * version = "1": This tells Cargo to use the latest version compatible with major version 1 (e.g., 1.28.1). This is a common practice that allows you to get non-breaking updates and bug fixes automatically. * features = ["full"]: This is an array of feature flags we want to enable. Here, we’re just enabling the one full feature.

You’ve now declared the core engine of our concurrent application. In the next tasks, we’ll add more specialized libraries that will run on top of the Tokio runtime we’ve just included.

Add reqwest for HTTP Requests

Mục tiêu: Add the reqwest crate to the project’s dependencies in Cargo.toml to enable the application to perform asynchronous HTTP requests for web scraping.

Excellent! In the previous task, you established the asynchronous foundation of our application by adding the tokio runtime. With tokio, our program has the engine to handle many tasks concurrently. Now, we need to give it the tools to perform its primary function: communicating with the web. This is where reqwest comes in.

The Role of an HTTP Client

At its core, web scraping is about fetching data from websites. To do this, our Rust program must behave like a web browser. It needs to send a request to a web server’s URL and then receive and interpret the server’s response, which is typically the HTML content of the page. The component responsible for handling this network communication is called an HTTP Client.

For our project, we’ll use reqwest, one of the most popular and powerful HTTP client libraries in the Rust ecosystem.

Why `reqwest`?

reqwest is an excellent choice for several reasons:

Ergonomic API: It provides a simple and intuitive interface for making web requests, which makes our code cleaner and easier to write and read.
Asynchronous by Default: reqwest is built on top of tokio and is designed for asynchronous operations out-of-the-box. This means it integrates perfectly with the runtime we just added, allowing us to make hundreds of non-blocking network requests concurrently. When we ask reqwest to fetch a URL, it won’t lock up our program; instead, it will work with tokio to let the CPU handle other tasks while waiting for the server’s response.
Feature-Rich: It comes with sensible defaults but also supports a wide range of features like custom headers, cookies, proxies, and more, which are essential for more advanced scraping tasks.

Adding `reqwest` to `Cargo.toml`

Just as we did with tokio, we will declare reqwest as a dependency in our Cargo.toml file. This tells Cargo to download and compile the reqwest crate when we build our project.

Open your Cargo.toml file and add the following line directly below your tokio dependency.

[package]
name = "concurrent_scraper"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "1", features = ["full"] }
reqwest = "0.12" # <-- Add this line

Let’s break down this addition:

reqwest = "0.12": This line tells Cargo to find and use a version of the reqwest crate that is compatible with 0.12 (e.g., 0.12.0, 0.12.1, etc.). Specifying the version ensures that your build is repeatable and won’t suddenly break if a future, incompatible version of the library is released.

By adding this single line, you have equipped our application with the ability to reach out to the internet and fetch the raw data we need to scrape. In the next tasks, we’ll add the tools to parse this raw HTML and extract meaningful information from it.

Integrate HTML Parser: Add the Scraper Crate

Mục tiêu: Add the scraper crate to your project’s dependencies to parse raw HTML. This will allow you to query the HTML structure using CSS selectors to extract specific data.

You are making excellent progress in setting up the foundation for our web scraper. So far, you’ve added tokio, the asynchronous runtime that will power our concurrent operations, and reqwest, the client that will fetch the raw content from web pages.

However, the data reqwest gives us is just a long, unstructured string of HTML code. To our program, it’s a jumble of characters like <div>, <p>, and <a>. We need a way to parse this string into a meaningful structure that we can navigate and query to find the specific pieces of information we’re after, like a product’s title or its price. This is where an HTML parser comes in.

From Raw Text to Structured Data: The Need for Parsing

Imagine you’re given a book and asked to find the title of Chapter 3. You wouldn’t read the entire book character by character. Instead, you’d use its structure—the table of contents, the chapter headings—to quickly locate the information. HTML parsing does the same for our code. It takes the raw HTML string and transforms it into a tree-like data structure known as the Document Object Model (DOM). Once we have this DOM tree, we can easily navigate it to find specific elements.

For this crucial task, we will use the scraper crate.

Introducing the `scraper` Crate

The scraper crate is a powerful and popular choice for parsing and querying HTML documents in Rust. It is built on top of html5ever, which is the same high-performance, browser-grade HTML parser used in Mozilla’s Firefox browser. This means it’s extremely robust and can handle the messy, imperfect HTML often found on real-world websites.

The standout feature of scraper is its use of CSS selectors for querying the DOM. If you’ve ever done any web development or used your browser’s developer tools, you’re likely familiar with CSS selectors. They are patterns used to select specific HTML elements. For example:

h1.main-title selects <h1> elements with the class main-title.
div#products selects the <div> element with the ID products.
a[href] selects all <a> (link) elements that have an href attribute.

Using CSS selectors makes extracting data incredibly intuitive and powerful, as we can precisely target the exact pieces of information we need.

Adding `scraper` to Your Project

Let’s add scraper to our project’s dependencies. Open your Cargo.toml file and add the scraper crate right below reqwest.

[package]
name = "concurrent_scraper"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "1", features = ["full"] }
reqwest = "0.12"
scraper = "0.19" # <-- Add this line

By adding scraper = "0.19", you are instructing Cargo to fetch and include version 0.19 (or any later compatible version within the 0.19.x range) of the scraper crate the next time you build the project.

With this addition, our core scraping toolkit is nearly complete. We now have:

tokio: The engine to run tasks concurrently.
reqwest: The tool to fetch raw HTML from URLs.
scraper: The tool to parse that HTML and find specific data within it.

In the next tasks, we will add the final pieces needed for data handling: a way to structure our scraped data and a library to write that data into a clean CSV file.

Storing Scraped Data by Adding a CSV Writer in Rust

Mục tiêu: Learn how to store data extracted from a web scraper into a CSV file. This task involves adding and configuring the ‘csv’ crate in a Rust project’s Cargo.toml file.

Storing Your Scraped Data: Adding a CSV Writer

You’ve done a fantastic job assembling the core toolkit for fetching and parsing web content. With tokio providing the asynchronous engine, reqwest handling the network requests, and scraper ready to parse the HTML, we have a clear path from a URL to the raw information we want. But what happens after we’ve extracted a product’s title and price? We need a way to store this data in a structured, usable format.

Simply printing the data to the console is useful for debugging, but the ultimate goal of our project is to create a clean, organized dataset. One of the most common and versatile formats for this purpose is CSV (Comma-Separated Values).

Why CSV?

A CSV file is a simple text file where each line represents a row of data, and values within that row are separated by commas. For our project, each row will represent a scraped product, and the columns will be the product’s details, like title, price, and url.

This format is an excellent choice because: * It’s Human-Readable: You can open a .csv file in any text editor and easily understand its contents. * It’s Universal: Virtually every data analysis tool, programming language, and spreadsheet application (like Microsoft Excel, Google Sheets, or Apple Numbers) can import and work with CSV files effortlessly. * It’s Lightweight: Being a plain text format, it’s very efficient in terms of storage.

To handle the creation of CSV files in a robust and efficient way, we will use the csv crate, which is the de facto standard for CSV handling in the Rust ecosystem.

Introducing the `csv` Crate

The csv crate provides everything we need for both reading and writing CSV data. It’s incredibly fast, memory-efficient, and designed with a strong focus on correctness. A key feature that we will leverage later is its seamless integration with the serde framework, which will allow us to automatically convert our Rust data structures directly into CSV rows without writing tedious manual conversion code.

Adding `csv` to Your Project Dependencies

Let’s add the csv crate to our Cargo.toml file. This will make its functionality available to our program. Open the file and add the new line for csv under the scraper dependency.

[package]
name = "concurrent_scraper"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "1", features = ["full"] }
reqwest = "0.12"
scraper = "0.19"
csv = "1.3" # <-- Add this line

By adding csv = "1.3", you are telling Cargo to download and compile version 1.3 (or a compatible newer version like 1.3.1) of the csv crate.

With this final dependency, our external toolkit is complete for now. We have all the necessary libraries to handle the entire lifecycle of our scraping process: from making a concurrent request to parsing the result and, finally, to saving the extracted data into a clean file.

Our next task will be to add one more small but crucial utility, serde, which will help us define the shape of our data inside our Rust program and bridge the gap between our code and the CSV file we want to create.

Add Serde Dependency for Data Serialization

Mục tiêu: Learn about serialization and add the serde crate with its derive feature to your Cargo.toml to facilitate converting Rust data structures into formats like CSV.

Bridging Your Code and Your Data: Introducing Serde for Serialization

You’ve successfully added all the major functional components to our project: an async runtime (tokio), an HTTP client (reqwest), an HTML parser (scraper), and a CSV writer (csv). This is a fantastic setup! However, there’s one crucial piece of glue missing. How do we easily get the data from our Rust code (which we will soon define in a struct) into the CSV file using the csv crate?

The answer is serialization.

What is Serialization?

At a high level, serialization is the process of converting an in-memory data structure—like a Rust struct that holds a product’s title and price—into a specific format for storage or transmission. In our case, we want to convert our product data into a CSV row (a string of text with commas). The reverse process, converting the formatted data back into an in-memory data structure, is called deserialization.

Without a framework to handle this, we would have to write tedious and error-prone code to manually build the CSV strings for each product. This is where serde comes in.

Meet `Serde`: The Powerhouse of Rust Data Handling

Serde (pronounced sur-day, for SERialization/DEserialization) is the definitive data handling library in the Rust ecosystem. It provides a powerful and generic framework for serializing and deserializing Rust data structures to and from various formats.

One of the most beautiful aspects of Serde is its ecosystem. The core serde crate provides the mechanisms, and other crates, like csv, serde_json (for JSON), and bincode (for a compact binary format), provide the specific format implementations. This means that by using Serde, you can make your data structures compatible with dozens of data formats with minimal effort.

The Magic of the `derive` Feature

The real power of Serde for application developers comes from its derive macro. Manually implementing the logic to serialize a struct would be repetitive. Instead, Serde allows us to simply add an attribute to our struct definition, like #[derive(Serialize)]. When we compile our code, this “derive macro” automatically generates the highly optimized serialization implementation for us. It’s a perfect example of Rust’s focus on making developers productive without sacrificing performance.

To use this feature, we must explicitly enable it when we add serde to our Cargo.toml.

Adding `serde` to Your Project

Let’s add serde with its derive feature to our Cargo.toml file. This is the final dependency we need for the initial setup.

[package]
name = "concurrent_scraper"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tokio = { version = "1", features = ["full"] }
reqwest = "0.12"
scraper = "0.19"
csv = "1.3"
serde = { version = "1.0", features = ["derive"] } # <-- Add this line

Let’s break down this addition: * serde = { ... }: We’re adding the serde crate. * version = "1.0": We specify the major version to ensure compatibility. * features = ["derive"]: This is the crucial part. We are telling Cargo that we need the serde crate compiled with its derive feature enabled, which gives us access to the #[derive(Serialize)] and #[derive(Deserialize)] macros.

Congratulations! Your project’s manifest is now fully configured with all the necessary external libraries. You have a complete toolkit at your disposal: * tokio for concurrency. * reqwest for fetching web pages. * scraper for parsing HTML. * csv for writing data to a file. * serde for easily converting your Rust data into the CSV format.

The foundation is now laid. In the next and final task of this step, we will compile the project to ensure Cargo can download and build all these dependencies correctly.

Compile the Rust Project with Cargo

Mục tiêu: Use the cargo build command to download, resolve, and compile all the project dependencies defined in the Cargo.toml file. This verifies the setup and prepares the environment for development.

Putting It All Together: Compiling Your Project

You have meticulously assembled the shopping list for our project. Your Cargo.toml file is now a complete declaration of all the external libraries—or crates—we need to build our powerful web scraper. You’ve included tokio for concurrency, reqwest for web requests, scraper for HTML parsing, csv for data output, and serde for seamless data serialization.

The manifest file is ready, but these dependencies currently exist only as lines of text. We need to instruct Cargo to perform its magic: to resolve, download, and compile these crates, and then compile our own application code against them. This crucial step verifies that our setup is correct and prepares our environment for development. The command for this is cargo build.

What `cargo build` Really Does

Running cargo build is a foundational part of the Rust development workflow. When you execute it for the first time after adding dependencies, it orchestrates a series of important actions:

Dependency Resolution: Cargo reads your Cargo.toml and analyzes your dependency requirements. It doesn’t just look at the crates you listed; it also looks at their dependencies (known as transitive dependencies). For example, reqwest itself depends on other crates for networking, SSL/TLS, and more. Cargo calculates a complete “dependency graph” of every single crate needed to build your project, ensuring all their version requirements are compatible with each other.
Locking Dependencies: Once a valid set of dependencies is found, Cargo records the exact versions of every crate in a new file called Cargo.lock. This file is critically important for reproducible builds. It ensures that every time you, or anyone else, builds this project, the exact same versions of the dependencies are used, preventing unexpected issues from library updates. You should commit this file to your version control (e.g., Git).
Downloading and Compiling: Cargo then downloads the source code for all these crates from the official crates.io registry. After downloading, it compiles them one by one. This is often the most time-consuming part. You will see many lines of output in your terminal as each crate is compiled. Don’t be alarmed if this takes a few minutes; it’s a one-time cost for each dependency. Subsequent builds will be much faster because Cargo cleverly caches these compiled artifacts.
Building Your Crate: Finally, after all dependencies are ready, Cargo compiles your own code from the src directory. Right now, this is just the default “Hello, world!” program.
Creating the Executable: The final compiled program (an executable file) is placed in the target/debug/ directory.

Running the Build Command

Now, open your terminal (making sure you are still inside the concurrent_scraper directory) and run the following command:

cargo build

You will see output that looks something like this. The specific packages and versions may differ slightly, but the pattern will be the same.

    Updating crates.io index
   Compiling proc-macro2 v1.0.81
   Compiling unicode-ident v1.0.12
   Compiling quote v1.0.36
   Compiling syn v2.0.60
   ... (many more lines) ...
   Compiling scraper v0.19.0
   Compiling reqwest v0.12.4
   Compiling tokio v1.37.0
   Compiling concurrent_scraper v0.1.0 (/path/to/your/project/concurrent_scraper)
    Finished dev [unoptimized + debuginfo] in 1m 30s

The most important line is the last one: Finished dev [unoptimized + debuginfo] .... If you see this without any red error messages, it means success! Your environment is correctly configured, all necessary libraries are downloaded and compiled, and your project is ready for you to start writing the actual scraping logic.

If you now look at your project directory, you’ll see a new target folder and a Cargo.lock file. The target folder contains all the intermediate build files and the final executable.

What’s Next?

With our project’s foundation firmly in place and all our tools at the ready, we can now shift our focus from setup to implementation. The next logical step is to define the shape of the data we want to collect. We will create a Rust struct to represent a single product, specifying the fields like title, price, and URL that we intend to scrape.