What Tammy Actually Is
Tammy is a data transformation and workflow orchestration platform. It handles ETL pipelines, schema migrations, and batch processing for teams that work with structured and semi-structured data. The core idea is straightforward: you define your transformations as code, deploy them, and Tammy manages execution, retries, and dependencies. I started using it about three years ago when my team was drowning in cron jobs and ad-hoc SQL scripts. We needed something that could handle dependencies without turning into a spreadsheet of manual checks. Tammy fit because it treats pipelines as first-class objects. You can version them, rollback them, and chain them together.
The Transformative Power Of Tammy Now A Comprehensive Overview
Setting up your first pipeline isn't complicated. You create a project directory, write a configuration file that describes your sources and targets, and then define the transformations in Python or SQL depending on your stack. Tammy picks up the config, validates it, and runs the job. Here is the part nobody tells you about the setup: the logging. By default, Tammy sends logs to stdout, which is fine for local development. In production, you need to route those to a centralized system immediately. I wasted two days chasing a silent failure because I hadn't configured log shipping before deploying to the cluster. Once I switched to stderr-based structured logging with JSON output, debugging became normal again.
How The Transformation Layer Works
The transformation engine supports incremental updates, which is where most of the value lives. Instead of rewriting entire tables every run, Tammy tracks row-level changes and applies only what is different. This requires you to have a reliable primary key and a timestamp column for change detection. Without both, the engine falls back to full refresh mode. Schema evolution is handled through a migration system that runs before each transformation. You declare schema changes in a migrations folder, and Tammy validates them against the target environment. If a migration would drop a column that downstream queries depend on, the pipeline fails before it starts. This is intentional. It prevents the kind of silent breakage that happens when someone alters a table and half the dashboard breaks overnight. One counter-intuitive thing I learned: idempotency matters more than you think. If a pipeline is not idempotent, every retry creates duplicate records or overwrites data you did not mean to touch. I had a case where a customer merge job produced duplicate customer records because the retry logic did not include an upsert strategy. The fix was wrapping the insertion logic in a MERGE statement keyed on customer_id instead of a simple INSERT. That alone cut our duplicate rate from 4% to zero.
Get the Full Details
Deployment And Scheduling
Tammy supports multiple deployment targets: local execution, Kubernetes clusters, and managed cloud environments. For small teams, the local runner is sufficient. It uses a lightweight scheduler based on APScheduler and stores state in a SQLite database. It works fine until you need horizontal scaling or high availability. When moving to a cluster, the configuration changes but the pipeline definitions stay the same. That is one of the design strengths. I migrated a setup from my laptop to a Kubernetes cluster and only had to update the runtime configuration. The actual transformation code did not change at all. Scheduling is done through cron-like expressions or event-based triggers. Event-based triggers are useful when you want to start a pipeline when a file lands in S3 or when a message appears in a queue. The catch is that event triggers require a listener service to be running. If that service crashes, your triggers stop firing and you will not know about it unless you have monitoring in place.
Common Pitfalls And Where It Breaks
Tammy is not a magic solution. It has real limitations. The biggest one is handling unstructured data. If your pipeline involves document parsing, image processing, or anything that is not row-oriented, Tammy is the wrong tool. It is built for tabular and semi-tabular data. Trying to force it into a computer vision workflow just creates more problems than it solves. Another limitation: dependency management across environments. If you have dev, staging, and production, Tammy does not provide built-in environment isolation. You have to manage that yourself through configuration and naming conventions. I solved this by prefixing all resource names with the environment identifier and using separate database schemas per environment. It works but it adds overhead to every pipeline definition. Performance tuning is also something you learn the hard way. Tammy does not automatically parallelize transformations within a single job. If your pipeline has five independent steps, you need to explicitly configure them as parallel tasks. Otherwise they run sequentially. I discovered this when a pipeline that should have taken ten minutes ran for over two hours. Splitting the independent steps into parallel branches brought the runtime down to eight minutes.
Alternatives To Consider
If Tammy does not fit your needs, there are other options. dbt is stronger if your work is mostly SQL-based and you do not need custom Python logic. Apache Airflow gives you more control but requires significantly more infrastructure. Prefect sits somewhere in between, with a lighter footprint and better developer experience than Airflow but more flexibility than Tammy. The right choice depends on your team size, your data volume, and how much custom logic your pipelines require. Tammy works well for mid-sized teams that need a balance between simplicity and capability. It is less suitable for teams that want maximum flexibility or for teams that primarily work with unstructured data. I have stuck with it because the incremental update engine saves us hours on every run. The migration system prevents accidental schema breaks. And the deployment model means we do not spend time maintaining orchestration infrastructure. Those are the things that matter when you are running dozens of pipelines every day.
