Code Modules: What They Are, Why We Built Them, and How They Work

Every data team eventually writes the same small helper function more than once. It might clean phone numbers, parse a date format a customer insists on, or wrap a call to an internal API. The function is tiny, and it is needed everywhere. On RapidCanvas, until recently, “everywhere” meant pasting that function into every place that used it.

Code Modules fix that. You write the helper once, as a proper Python package inside your project, and every recipe, prediction service and notebook in that project can import it. This post covers what a Code Module is, how people use one, and what happens between uploading a file and a recipe running import my_module.

TL;DR

• A Code Module is a reusable Python package that belongs to a project. Any recipe, prediction service or notebook in that project can import it with no setup code.
• Module details live in the platform database, files live in cloud storage, and uploads go straight from the browser to storage through short-lived signed URLs.
• Interactive runs always get the latest version of a module; scheduled jobs run on a snapshot taken when the schedule is published, so daytime edits never leak into production.

The problem: one helper, four copies

Take a data scientist, Priya, working on a customer-churn project. She has a function called clean_phone_numbers() that strips spaces, adds country codes and throws away junk values. Two recipes need it, the prediction service that scores new customers needs it, and so does the notebook where she explores the data.

So she pastes it into all four.

A month later a customer sends numbers in a new format and the function has to change. Priya updates it in Recipe A and in the notebook. She forgets Recipe B. The prediction service still has a version from two months ago. Nothing crashes. The numbers are just cleaned three slightly different ways, and the churn model quietly gets worse.

If you are not an engineer, think of a restaurant kitchen. Every cook has a hand-copied version of the house sauce recipe on a sticky note. When the head chef changes the recipe, some notes get updated and some don’t, and customers get a different sauce depending on who is cooking. A Code Module is the single laminated recipe card on the wall that everyone cooks from.

A few RapidCanvas terms come up throughout this post:

What we needed

A shared module had to meet a few constraints to be safe to use everywhere:

• No drift. One fix has to reach every consumer, without anyone tracking down copies.
• No setup code. A recipe should not have to fetch or install its own helpers. If reuse costs effort, people go back to pasting.
• Stable production. Scheduled jobs should not change behaviour mid-day because someone edited a helper.
• Large uploads without load on the API. Moving big files should not tie up the servers that handle everyone else’s requests.
• Survive project copies. When a project is copied, imports have to keep working in the copy.
• Isolation. A module stays inside its own project and customer.

What a Code Module is

A Code Module is a reusable Python package that belongs to a project. It has four parts:

• A name, such as text_utils. This is exactly what your code will import, so it must be a valid Python name: letters, numbers and underscores, not starting with a number.
• A description, so teammates know what it is for.
• Requirements (optional): any extra pip packages the module needs, such as phonenumbers==8.13.
• Files and folders: your .py code plus any supporting .md, .json or .yaml files, arranged however you like.

Once the module exists, any recipe, prediction service or notebook in the project can do this:

from text_utils.phones import clean_phone_numbers

No setup code and no copy-pasting. The platform makes sure the module is there before your code starts.

Using a Code Module

From a user’s point of view, the journey is short:

1. Create. Open the project’s Code Modules page and click Create Code Module. Give it a name, a description and any requirements, then drag in your files. The upload organiser lets you create folders and move files around before anything is saved.
2. Browse. Click a module to open its viewer, with a file tree on the left and the code on the right.
3. Maintain. Add or replace files, delete ones you no longer need, and edit requirements as the module grows.
4. Download. Get the whole module back as a zip file whenever you need it locally.

Engineers who prefer the terminal get the same thing through the rc CLI. A pipeline’s dag.yaml can list its code_modules, rc export pulls them down to a laptop, and local runs set up the Python path so the same import lines work offline. Deploys from CI go one step further: they compare file hashes and only re-upload a module when something in it actually changed.

How we built it

Two homes: the details and the code

A Code Module lives in two places, each suited to its job.

• The platform database stores the module’s details: its name, description, requirements, which project it belongs to, whether it is active, and where its files are.
• Cloud storage (the same object storage we use for project data) holds the actual files, in a dedicated location per customer and per project.

It is the same idea as a library catalogue and the shelves. The catalogue card tells you the title and where to find the book; the book itself sits on the shelf. The database is fast to search and filter, and storage is built to hold files of any size. Neither has to do the other’s job.

Uploads go straight to storage

When you drop files into the upload window, they don’t pass through our API servers at all.

This happens in three steps:

1. Ask for upload links. The browser tells the platform which files it wants to upload, and the platform creates a signed URL for each file.
2. Upload directly. The browser sends each file straight to cloud storage using those links.
3. Register the module. The browser asks the API to create the module. The API confirms the files have really arrived and only then saves the module’s details to the database.

A signed URL works like a valet key: it opens one specific door, for one specific file, and expires shortly. The browser gets exactly the access it needs and nothing more, and large uploads don’t tie up the servers that handle everyone else’s requests.

The API also checks everything before it saves anything. Module names must be valid Python. File paths can’t use tricks like ../ to escape the module’s own folder. Only the expected file types are accepted.

Delivery at run time

You never tell a recipe to “load” a module; it is simply there.

When you run a recipe (the same applies to starting a prediction service or a notebook):

1. Find the modules. The platform asks the database for every active Code Module in the project, with its storage location and requirements.
2. Pass them along. That list travels with the run request to the Python runtime inside the project’s environment.
3. Install packages. The runtime combines the module’s requirements with those of the recipe and any model. If two of them ask for different versions of the same package, the recipe’s choice wins, then the model’s, then the module’s. Whoever is closest to the actual code gets the final say.
4. Copy the code in. Each module is downloaded into the environment’s site-packages folder, the standard place Python looks for installed packages. Any old copy is wiped first, so you always get the current version. The runtime also adds the __init__.py marker files Python needs to treat every folder as part of the package, so you don’t have to remember them.
5. Run. Your recipe starts, and import text_utils just works.

To Python, a Code Module looks exactly like any package installed with pip. That is the trick: your code doesn’t need to know it is special.

Scheduled jobs get a snapshot

Interactive runs always use the latest version of a module. That is what you want while developing: change a file, run the recipe, see the result.

Scheduled jobs are different. A nightly pipeline that feeds a customer’s morning dashboard should not change its behaviour because someone edited a helper at 11 PM.

So when a schedule is published, the platform takes a snapshot. It copies every active module, with its requirements, into a location that belongs to that schedule alone. Every scheduled run then uses the snapshot, so you can keep experimenting with the live module during the day without touching production. When you want the scheduled job to pick up your changes, you republish it and a fresh snapshot is taken.

It is like a printed menu. The kitchen can test new dishes all day, but customers order from the menu printed this morning until a new one is printed.

Modules travel with the project

When a project is copied (from a sample project, between workspaces, or through export), its Code Modules come along. Their files are copied to the new project’s storage location, and each module keeps exactly the same name. That last detail matters: every recipe that says import text_utils keeps working in the copy without anyone touching a line of code.

Safe by default

A few guardrails apply everywhere:

• Project-scoped. A module belongs to one project and is visible only inside it.
• Customer-isolated. Every lookup is automatically limited to the current customer, and stored files are separated by customer and project.
• Same permissions as the project. If you can edit the project, you can manage its modules. If you can only view it, you can only view them.

Priya’s bug fix, revisited

Priya moves clean_phone_numbers() into a Code Module called text_utils, adds phonenumbers as a requirement, and replaces the four pasted copies with one line each:

from text_utils.phones import clean_phone_numbers

A month later the new phone format arrives. She uploads the fixed phones.py to the module, once.

• The next time Recipe A and Recipe B run, the platform copies in the new text_utils. Both recipes now clean numbers the same way.
• The prediction service picks up the new version the next time it starts.
• Her notebook gets it on its next start as well.
• The nightly scheduled pipeline keeps running on its snapshot, unaffected while she tests the change during the day. Once she is happy, she republishes the schedule and the fix goes into production.

One edit, and every consumer is consistent.

Key takeaways

Keep code in one place, and bring it to wherever it runs. Rather than asking every recipe to fetch its dependencies, the platform delivers them automatically, so reuse costs nothing.

*Separate what something is from what it contains.* The database holds the small, searchable details. Cloud storage holds the files. Signed URLs let the browser move the heavy files without burdening the API.

Live for experimentation, snapshots for production. Interactive runs always get the newest code. Scheduled work gets a stable, frozen copy until someone deliberately republishes it.

Try it

Open any project, go to Code Modules, and turn your most-copied helper into your first module. The user guide is at docs.rapidcanvas.ai/basic/projects/code-modules, and if you want to talk through your own pipelines, book a discovery call.