Polars for Data Engineering: Faster DataFrames in Python

Create a multidimensional data frame in Polars, building a dataset with serial numbers, names, date of birth, date time, and location, then print the data frame.

Explore viewing data in Polars data frame using head, tail, sample, and describe to display records and summarize statistics such as count, mean, std, min, max, and percent values.

Master Polars select expressions to retrieve and exclude columns, using name or data type filters with contains, matches, and regular expressions.

Learn to rename and alter Polars dataframes by converting column names to uppercase, prefixing or suffixing names, and adding new prefixed columns using dot name, dot map, and lambda.

Explore handling nulls in Polars by creating a data frame with missing values, counting nulls, checking is_null, and filling nulls with strategies like forward, backward, minimum, maximum, and zero.

Use Polars to apply filter and transformations, select all columns, and filter by country code equals us; print the resulting data frame showing three entries.

Learn to sort records in Polars data frames by salary and country code, using select, sort by, and handle ascending, descending, and nulls last.

Import Polars, load and inspect the dataset, then perform aggregation by counting records and computing min, max, and sum of the salary column, plus retrieving the first value.

Explore advanced aggregations in Polars by grouping by country code and aggregating salary and first name, applying having conditions and filters. Count users per group across US and UK.

Explore joining data frames with polars, including inner, left, outer, cross, semi, antijoin, and asof joins, using customer and order data and time-based matching with tolerance.

Learn to unpivot a Polars data frame from wide to long format using melt, with id columns and resulting variable and value columns.

Learn how to use window functions in Polars to compute counts and maxima by partitions such as country code and salary, including expressions like salary modulo two.

Explore query planning in polars for data engineering using the explained method to compare lazy frame transformations, including adding a column, uppercasing it, and filtering by country code us.

Explore reading and writing JSON with Polars, parsing JSON into data frames. Convert to structs and generate JSON files, including newline-delimited data and lazy API concepts using scan.

Read multiple csv files matching a regex like employees_*.csv using Polars for data engineering, load them into a data frame, and print all retrieved data.

Write the dataset into multiple csv files using a range of five, appending numerical values across files and generating five distinct outputs.

Master parallel processing by reading multiple files from the source with a for loop, then combine them into a single Polars data frame using collect all.

Create a script to read data from a Postgres database using SQLAlchemy by establishing a connection, querying the employees table, and printing the results.

Initialize sql context and register datasets to manage sql queries for data frame and lazy frame identifiers. Import Polars and print to confirm setup.

Register data and lazy frames in the Polars sql context, using globals, identifiers, or keyword arguments, and learn to bring pandas frames into sql context and collect lazy results.

Register a data frame, lazy frame, and pandas data frame as polars tables and convert the pandas frame to a polars data frame. Run show tables to display registered tables.

Create a new table from the results of a joined SQL query, naming the table and printing the data; the defined table can be used for further operations.

Learn to use common table expressions with clause in Polars, create a lazy frame, register it in a SQL context, and query where age is greater than 20.

Read flat files directly in Polars without registering them in a SQL context, and execute a direct CSV read and query to print results.