Column Operations
When faced with a new dataset, how do you quickly isolate the exact pieces of information you need, or prepare a new feature for analysis, without disrupting everything else? Pandas DataFrames are powerful, but their true utility comes from efficiently manipulating their structure. Understanding how to select, create, and modify columns is a core skill for any data professional. These operations allow you to refine your raw data into a clean, analysis-ready format, making your workflow significantly more efficient.
Selecting Columns: Your Data's Address Book
Accessing specific columns is the first step in almost any data manipulation task. You can think of column selection as looking up an entry in your data's address book. When you select a single column from a DataFrame, Pandas returns a Series, which is a one-dimensional labeled array capable of holding any data type. This Series retains the original DataFrame's index, allowing for easy alignment with other data.
orders_df from the previous example, select the customer_id column and print its type. What type of object is returned?To select multiple columns, you pass a list of column names to the DataFrame. This is a key distinction from single column selection. When you provide a list, Pandas returns a new DataFrame containing only the specified columns. This preserves the two-dimensional structure, which is often desired when working with subsets of your data.
df[['col1', 'col2']] to select multiple columns, but single brackets df['col1'] for one column?Crafting New Columns: Expanding Your Dataset
Adding new columns is a common step in data preparation, allowing you to enrich your dataset with calculated values or new categories. You can add a new column by assigning either a single scalar value that will be broadcast across all rows, or by assigning a Series (often derived from existing columns) that aligns with the DataFrame's index. This flexibility makes it easy to create features for analysis or add metadata.
shipping_cost to orders_df. Assume shipping_cost is 5% of the total_amount for orders over 100.00, and a flat 5.00 for orders 100.00 or less. You'll need to create the total_amount column first, as shown in the previous example.Updating Existing Columns: Refining Your Data
Modifying an existing column means changing its values in place. This is achieved using the same assignment syntax as creating a new column, but you target a column name that already exists in the DataFrame. Pandas will overwrite the old values with the new ones, directly altering your DataFrame. This is useful for data cleaning, standardization, or applying transformations to existing features.
Single column selection uses
df['column_name']and returns a Pandas Series.Multiple column selection uses
df[['col1', 'col2']]and returns a DataFrame.New columns can be added by assigning a scalar value (broadcast) or a Series (derived from existing columns).
Existing columns are modified by re-assigning values to their name, directly updating the DataFrame.
These fundamental column operations are essential for data cleaning, feature engineering, and quickly isolating the specific data you need for analysis.