Module 15
Combining files and parameters
Combine folder-like file packs with one reusable transformation.
You'll be able to
- Combine many files
- Keep the source file name
- Parameterise a query
Lessons
- 1.Combine files from a folderApply one parsing step to every file in a folder and stack the results.
- 2.Turn a sample-file transformation into a functionBuild and test a reusable file-parsing function, as Combine Files does.
- 3.Parameters: one value, many queriesReplace repeated hard-coded values with parameters that queries reference.
- 4.Combine files whose columns differCombine files with drifting schemas without losing columns or rows.
Take the combining files and parameters quiz (8 questions)
Exercises
- intermediate~12 minPro
Combine every monthly CSV in a folder (kite shop)
Read every file in the folder, keep the file name, and stack the orders into one typed table.
- intermediate~12 minPro
Combine every monthly CSV in a folder (bakery)
Read every file in the folder, keep the file name, and stack the orders into one typed table.
- intermediate~12 minPro
Combine every monthly CSV in a folder (bike rental)
Read every file in the folder, keep the file name, and stack the orders into one typed table.
- intermediate~10 minPro
Combine only the CSV files in a mixed folder (kites)
The folder also holds a README and a notes file. Combine just the .csv files, keeping the file name.
- intermediate~10 minPro
Combine only the CSV files in a mixed folder (bakery)
The folder also holds a README and a notes file. Combine just the .csv files, keeping the file name.
- advanced~12 minPro
Combine files that each start with banner lines (kites)
Every export starts with two banner lines before the header. Write one transform for a file and apply it to all of them.
- advanced~12 minPro
Combine files that each start with banner lines (bakery)
Every export starts with two banner lines before the header. Write one transform for a file and apply it to all of them.
- advanced~11 minPro
Combine files whose columns come in different orders (kites)
Different branches export the same three columns in different orders. Combine them by column name.
- advanced~11 minPro
Combine files whose columns come in different orders (bakery)
Different branches export the same three columns in different orders. Combine them by column name.
- intermediate~10 minPro
Take each file's month from its name (kites)
The files don't contain a date column, but their names do (orders-2026-03.csv). Add a Month date to every row.
- intermediate~10 minPro
Take each file's month from its name (bakery)
The files don't contain a date column, but their names do (orders-2026-03.csv). Add a Month date to every row.
- advanced~12 minPro
Combine files when newer ones gain a column (kites)
From March the export added a Discount column. Combine every file and keep Discount, null for older files.
- advanced~12 minPro
Combine files when newer ones gain a column (bakery)
From March the export added a Discount column. Combine every file and keep Discount, null for older files.
- intermediate~9 minPro
Load only the most recently saved file (kites)
Each month a new snapshot lands in the folder; only the newest one matters. Load it, whatever it's called.
- intermediate~9 minPro
Load only the most recently saved file (bakery)
Each month a new snapshot lands in the folder; only the newest one matters. Load it, whatever it's called.
- intermediate~10 minPro
Combine a folder of JSON files (marketplace)
A partner drops one JSON file per day, each a list of order records. Combine them all with the file name.
- intermediate~10 minPro
Combine a folder of JSON files (wholesale)
A partner drops one JSON file per day, each a list of order records. Combine them all with the file name.