Define a Public Product Struct in Rust

Mục tiêu: Define a public, empty struct named Product in src/main.rs. This serves as the initial step in creating the data model for a web scraper application.

With your project environment now fully configured and all necessary dependencies downloaded and compiled, you’ve built a solid launchpad for our application. That successful cargo build was the confirmation that your toolkit is ready. Now, it’s time to shift from setup to implementation and begin writing the core logic of our scraper.

The very first step in any data-oriented program is to define the shape, or structure, of the data you’re working with. Before we can even think about fetching a web page or parsing HTML, we need a clear, well-defined blueprint for the information we intend to capture. In Rust, the primary way to create a custom data type like this is by using a struct.

Understanding Structs in Rust

A struct, short for structure, is a custom data type that lets you name and package together related values that make up a meaningful group. Think of it as a template for a piece of data. If we’re scraping products, we can create a Product struct that will hold all the information relevant to a single product, such as its title, price, and the URL where we found it.

Using structs provides two major benefits:

Clarity and Organization: It bundles related data together. Instead of juggling separate variables like product_title, product_price, etc., we can have a single product variable that contains all of this information. This makes our code much cleaner and easier to reason about.
Type Safety: By defining a Product type, we can leverage Rust’s powerful compiler. The compiler will ensure that we can’t accidentally use, for example, a number where a product is expected. This helps catch a whole class of bugs at compile time, long before they can cause problems in a running application.

Defining Your First Struct

Our current task is to define the shell for our Product data. We will do this in src/main.rs. We’ll also make it public by using the pub keyword.

Why pub?

The pub keyword stands for “public,” and it controls the visibility of an item. In Rust, all items (structs, functions, modules, etc.) are private by default, meaning they can only be accessed from within the module where they are defined. While our entire application currently lives in main.rs, we will eventually refactor it into separate modules (e.g., scraper.rs, cli.rs) for better organization. By marking our Product struct as pub now, we are ensuring that it will be accessible from these other modules later on. It’s a best practice that sets us up for a more maintainable project structure.

Open your src/main.rs file and add the Product struct definition above the main function. For now, the struct will be empty; we will add its data fields in the very next task.

// src/main.rs

// This is the blueprint for the data we want to scrape.
// The `pub` keyword makes this struct visible to other modules in our project,
// which will be important when we refactor our code later. In Rust, types
// like structs are typically named using UpperCamelCase.
pub struct Product {
    // We will add the fields that make up a Product (like title, price, etc.)
    // in the next task. The curly braces define the body of the struct.
}

fn main() {
    println!("Hello, world!");
}

You have now successfully defined a new custom type named Product. While it doesn’t hold any data yet, you’ve created the foundational data structure that will be at the heart of our web scraper. Every item we scrape from the e-commerce site will be turned into an instance of this Product struct.

What’s Next?

With the shell of our Product struct in place, the next logical step is to give it the fields that will hold our scraped data. In the upcoming tasks, we will populate this struct with fields for the product’s title, price, and URL, and then add powerful attributes that will enable us to easily debug our data and serialize it into a CSV file.

Populate a Rust Struct with Fields

Mục tiêu: Add title, price, and url fields of type String to the Product struct. This task focuses on defining the data structure for a web scraping application and explains the reasoning behind using the String type and public visibility.

In the previous task, you successfully laid the groundwork by defining the Product struct’s empty shell. You created a new, custom type that will be central to our application. Now, it’s time to give that shell its substance by defining the specific pieces of data each Product will hold.

Populating the `Product` Struct

A struct is like a blueprint. An empty blueprint isn’t very useful; we need to add the rooms and dimensions. For our Product struct, this means adding fields. Each field represents a single piece of information we want to scrape from the e-commerce website. Based on the project’s goals, we need to capture three key details for each product: its title, its price, and the specific URL for its product page.

We will add these fields to our struct, each with a name and a type.

title: String: This will hold the product’s name. We use the String type because a product title is text, and String is Rust’s standard, owned, growable string type. This is perfect for holding text data that we’ve extracted from an HTML document.
price: String: You might wonder why we’re using String for a price instead of a number type like f64. This is a common and robust practice in web scraping. Prices on websites often contain currency symbols (e.g., “$”, “£”), commas, or even text (e.g., “Contact for price”). Storing the price as a string exactly as it appears on the page is the most reliable way to capture it. We can always parse it into a number later if we need to perform calculations.
url: String: This field will store the direct link to the product’s detail page. Storing the URL is good practice as it provides a reference back to the source of the data and can be used to scrape more details later if needed.

Let’s update the src/main.rs file to include these fields inside the Product struct’s curly braces.

// src/main.rs

pub struct Product {
    // Add the `title` field to hold the product's name as an owned String.
    pub title: String,
    // Add the `price` field. We use String to robustly capture text and symbols.
    pub price: String,
    // Add the `url` field to store the link to the product page.
    pub url: String,
}

fn main() {
    println!("Hello, world!");
}

Note: We are also making the fields pub (public) for the same reason we made the struct public. This will allow code in other modules (which we will create later) to access and modify these fields when creating Product instances.

Understanding Owned Data with `String`

It’s important to understand why we’re using String and not its counterpart, &str (a “string slice”).

&str is a borrowed type. It’s a view into some string data that exists somewhere else. It doesn’t own the data itself.
String is an owned type. It allocates memory on the heap and is responsible for that data.

When we scrape a web page, we’ll parse an HTML document, find the text for a title, and create a Product instance. The original HTML document will be discarded after parsing. If our Product struct used &str, it would be borrowing text that is about to be destroyed, which the Rust compiler would correctly prevent. By using String, our Product struct takes full ownership of the scraped text, copying it into its own memory. This allows each Product instance to be a self-contained, independent piece of data that can be passed around our program, sent between threads, and eventually written to our CSV file.

You’ve now defined a clear, structured blueprint for the data our application will collect. This is a critical step that brings us closer to writing the actual scraping logic.

What’s Next?

With our Product struct now defined with the correct fields, the next step is to give it some “superpowers” using Rust’s attribute system. We’ll add a derive attribute that will automatically implement traits to make our struct printable for debugging and, most importantly, serializable so the csv crate can easily write it to a file.

Enhance a Rust Struct with `derive` for Debugging and Serialization

Mục tiêu: Learn how to use the #[derive] attribute in Rust to automatically implement the Debug trait for easy printing and the serde::Serialize trait to prepare a Product struct for CSV serialization.

Excellent! In the previous task, you defined the precise data fields our Product struct will hold: title, price, and url. Your struct is now a perfect blueprint for a product. However, in its current state, it’s just a passive container for data. To make it truly useful within our application, we need to grant it some essential behaviors. We will achieve this using one of Rust’s most powerful and developer-friendly features: the derive attribute.

Giving Your Struct Superpowers with `derive`

Right now, if we tried to print an instance of our Product struct to the console to see what’s inside, the compiler would give us an error. Similarly, if we tried to hand it over to the csv crate and say “write this to a file,” the crate wouldn’t know what to do. Our struct doesn’t yet implement the necessary “traits” (which are like interfaces or contracts in other languages) that define these behaviors.

We could implement these traits manually, writing out all the boilerplate code to specify how a Product should be formatted for debugging or converted into a series of bytes for serialization. However, Rust provides a much better way: #[derive(...)].

#[derive(...)] is a special kind of Rust attribute. An attribute is metadata you add to your code to modify its behavior. The derive attribute, specifically, is a procedural macro that instructs the Rust compiler to automatically generate the code to implement certain common traits for us at compile time. This is a cornerstone of idiomatic Rust, saving us from writing repetitive code while still producing highly optimized implementations.

We will derive two crucial traits for our Product struct:

Debug: This trait comes from Rust’s standard library. Implementing it allows our struct to be printed in a developer-friendly format using the {:?} (debug) or {:#?} (pretty-debug) format specifiers in println!. This is an indispensable tool for inspecting the data you’ve scraped to make sure everything is working as expected.
serde::Serialize: This is the powerhouse trait from the serde crate we added earlier. The name “Serialize” means to convert a data structure into a format suitable for storage or transmission. By deriving Serialize, we are automatically generating the code that teaches serde how to transform a Product instance into a format that the csv crate can understand and write as a row in a file. The csv writer will expect the header row to match the struct field names: title, price, url.

Let’s update src/main.rs to add this attribute directly above the struct definition.

// In src/main.rs

// This is an attribute. It tells the Rust compiler to automatically generate
// implementations for the traits listed inside the parentheses.
#[derive(Debug, serde::Serialize)]
pub struct Product {
    pub title: String,
    pub price: String,
    pub url: String,
}

fn main() {
    println!("Hello, world!");
}

By adding this single line, #[derive(Debug, serde::Serialize)], you have equipped your Product struct with two vital capabilities:

It can now be easily printed for debugging.
It is now fully compatible with the csv crate’s serialization mechanism.

This completes the definition of our core data structure. We now have a robust, self-contained, and highly capable blueprint for the data we will be scraping.

What’s Next?

With our data structure fully defined and ready to go, we have completed this crucial step. We are now perfectly positioned to move on to the core of the project: writing the asynchronous function that will connect to a website, download its HTML, and parse it to extract the data needed to create instances of our newly defined Product struct.

Derive Debug and Serialize for a Rust Struct

Mục tiêu: Enhance the Product struct by using the #[derive(Debug, serde::Serialize)] attribute. This will automatically implement the Debug trait for easy printing and debugging, and the serde::Serialize trait to enable writing instances to formats like CSV.

Giving Your Struct Superpowers with `derive`

Right now, if we tried to print an instance of our Product struct to the console to see what’s inside, the compiler would give us an error. Similarly, if we tried to hand it over to the csv crate and say “write this to a file,” the crate wouldn’t know what to do. Our struct doesn’t yet implement the necessary traits (which are like interfaces or contracts in other languages) that define these behaviors.

#[derive(...)] is a special kind of Rust attribute. An attribute is metadata you add to your code to modify its behavior. The derive attribute, specifically, is a procedural macro that instructs the Rust compiler to automatically generate the code to implement certain common traits for us at compile time. This is a cornerstone of idiomatic Rust, saving us from writing repetitive code while still producing highly optimized implementations.

We will derive two crucial traits for our Product struct:

1. The `Debug` Trait

The Debug trait comes from Rust’s standard library and is your best friend during development. Its entire purpose is to provide a developer-friendly, human-readable representation of a type.

What it does: Implementing Debug allows our struct to be printed to the console using the {:?} (debug) or {:#?} (pretty-print debug) format specifiers in macros like println!.
Why it’s essential: When you’ve scraped a web page and created a Product instance, how do you know if you extracted the right data? You print it! The Debug trait makes this trivial. It is an indispensable tool for inspecting the state of your variables and ensuring your logic is working as expected. Without it, debugging would be significantly more difficult.

2. The `serde::Serialize` Trait

This is the powerhouse trait from the serde crate we added earlier. The name “Serialize” means to convert a data structure from its in-memory representation (like our Rust struct) into a specific wire format suitable for storage or transmission (like a CSV row).

What it does: By deriving Serialize, we are automatically generating the code that teaches serde how to transform a Product instance into a format that the csv crate can understand.
Why it’s essential: This is the magic that connects our Rust code to our final output file. Instead of manually building a comma-separated string like "Some Product, $19.99, http://...", which is tedious and prone to errors (e.g., what if a title contains a comma?), we can simply hand our Product instance to the csv writer. Because our struct implements Serialize, the csv crate knows exactly how to convert it into a valid CSV row. As a bonus, the csv writer will use the struct’s field names (title, price, url) to automatically write the header row of the CSV file.

Let’s look at the final code in src/main.rs. By adding this single attribute, you’ve made your data structure incredibly capable.

// In src/main.rs

// This is an attribute. It tells the Rust compiler to automatically generate
// implementations for the traits listed inside the parentheses.
// - `Debug`: Allows us to print the struct to the console for debugging using `println!("{:?}", ...);`.
// - `serde::Serialize`: Allows the struct to be converted into other data formats,
//   which is essential for the `csv` crate to write it to a file.
#[derive(Debug, serde::Serialize)]
pub struct Product {
    // These fields are public so they can be accessed from other parts of our
    // application, which will become important as our project grows.
    pub title: String,
    pub price: String,
    pub url: String,
}

fn main() {
    println!("Hello, world!");
}

This completes the definition of our core data structure. We now have a robust, self-contained, and highly capable blueprint for the data we will be scraping.