Running on TensorFlow?

TensorFlow CUDA mismatch or GPU OOM? We sort it out.

TensorFlow is still powering a large share of production ML, especially anything with a long history or a TPU investment behind it. The problems rarely come from the model code itself. They come from CUDA and cuDNN versions that disagree, an OOM that fires on the second run but not the first, or a SavedModel that no longer loads under TensorFlow 2.17. We work through the environment, the graph and the serving layer until training is stable and inference is deterministic.

  • Since 2015Fixing technology
  • FreeEstimates
  • US teamReal engineers

tensorflow.log

still broken

What it is

TensorFlow at a glance

TensorFlow is an open-source machine learning framework built at Google Brain and released in November 2015. It provides low-level numerical operations, high-level Keras APIs for model building, SavedModel for serialization, TensorFlow Serving for inference, TensorFlow Lite for mobile and embedded, and TensorFlow.js for browser-side use. TensorFlow 2 (2019) made eager execution the default and consolidated Keras as the primary user-facing API.

The framework runs on CPU, NVIDIA and AMD GPUs through various backends, Google TPUs, Apple Silicon through tensorflow-metal, and distributed clusters through tf.distribute. It is open source under the Apache 2.0 license. Google maintains the project but community adoption has shifted substantially toward PyTorch for new research work, while TensorFlow remains dominant for production inference, mobile deployment and TPU-based training.

Type
Machine learning framework
First released
2015
License
Apache 2.0, open source
Maintained by
Google and contributors
Typically used for
Training neural networks, production inference, mobile ML with TF Lite, TPU workloads, browser-side ML via TensorFlow.js

The honest version

TensorFlow pros and cons

No technology is all upside. This is where TensorFlow earns its place and where it tends to cause trouble.

Where TensorFlow is strong

  • Production tooling is matureSavedModel, TensorFlow Serving and TF Lite together cover training, server inference and mobile deployment under one umbrella, with versioning, warmup and batching handled out of the box.
  • TPU support is first classIf you have access to Google Cloud TPUs, TensorFlow is the best-supported path. JAX is catching up, but for turnkey distributed training on TPU pods the ecosystem is still here.
  • Keras is a pleasant modeling APISequential, functional and subclassed models cover almost every architecture, with sensible defaults for callbacks, metrics and checkpointing without leaving the framework.
  • Mobile and browser deploymentTensorFlow Lite runs on Android, iOS and microcontrollers. TensorFlow.js runs the same model in a browser. Few other frameworks give you both targets with a shared training story.
  • Graph mode for inference performancetf.function compiles a Python function into a graph that can be exported, optimized, quantized and served without a Python interpreter in the hot path.
  • Mature ecosystem around itTensorFlow Hub, TensorFlow Datasets, TensorFlow Probability, TFX for pipelines and a long bench of public tutorials all still get maintained updates.

Where TensorFlow bites

  • CUDA and cuDNN version painA TensorFlow build is pinned to specific CUDA and cuDNN versions. Install the wrong pair and the GPU is invisible. The install guide is correct; most users read it only after the first failure.
  • Community momentum has moved to PyTorchNew research papers, new tutorials and new graduate students default to PyTorch. Hiring someone who prefers TensorFlow is possible, but the pool is older and narrower.
  • tf.function retracing is a hidden costA model that retraces on every call because of a changing input shape or a Python side effect can silently cost an order of magnitude in throughput without any error.
  • SavedModel compatibility can biteA model saved under TensorFlow 2.12 does not always load cleanly under 2.17. Custom layers, lambda layers and non-serializable closures are the common breakers.
  • OOM errors are not always honestTensorFlow grabs most of the GPU memory by default. The first script that runs after a kernel restart succeeds, the second fires OOM. The allocator setting is the usual fix.
  • Old tf.contrib code is unmovableCodebases that pre-date TensorFlow 2 often depend on contrib modules that no longer exist. Porting them is not a weekend project; rewriting the model is sometimes cheaper.

Sound familiar?

Common TensorFlow problems we fix

The TensorFlow problems that bring people to us, and what you have once each one is fixed.

OOM when allocating tensor

Training runs out of GPU memory

A batch size tuned on a bigger card, a model that holds activations longer than it should, or a second notebook that already reserved most of the GPU through TensorFlow's greedy allocator.

Memory budgeted, batch sized honestly

Could not load dynamic library 'libcudnn.so.8'

CUDA and cuDNN versions do not match

A TensorFlow build expecting one CUDA and cuDNN pair while the system has another, often after a driver update or a conda environment that pulled in a newer stack than TensorFlow supports.

Stack pinned and documented

Retracing function. Expensive if repeated.

tf.function retraces on every call

An input whose shape changes per batch, a Python-level operation inside the decorated function, or a model that was never warmed up with a representative input.

Retracing eliminated, throughput restored

Unable to load SavedModel

SavedModel refuses to load

A custom layer, lambda, or activation that was saved under a signature TF no longer recognizes, or a version gap between the saving and loading environments.

Model loads under the target version

tf.function tried to create variables on non-first call

Variables created inside tf.function

A pattern that works eagerly but fails once the function is compiled, because variables must be created exactly once outside the traced call path.

Variable creation moved out of trace

module 'tensorflow' has no attribute 'contrib'

Legacy TF1 code does not run on TF2

A codebase relying on tf.contrib or tf.Session, which no longer exist in TensorFlow 2. Compatibility shims cover some of it; the rest needs porting.

TF1 code either ported or shimmed

cuDNN launch failure

cuDNN kernels fail at run time

A GPU under thermal throttling, a driver that just updated under the user's feet, or a batch shape the chosen cuDNN algorithm cannot handle on the installed version.

Kernels stable across the training run

assertion failed: predictions must be >= 0

Numerical assertions trip in the graph

An activation that produced NaN or a negative probability, often from a learning rate too high or a loss that is not numerically stable on fp16.

Numerics stable, assertions silent

What we do

TensorFlow help, from one OOM to a managed stack

Call us about a single OOM or SavedModel issue, or let us keep the whole TensorFlow stack healthy.

Rescue
A training run that fires OOM, a SavedModel that no longer loads, or a serving endpoint that started returning garbage. We stabilize before we explain.
CUDA and environment setup
Pinned CUDA, cuDNN, driver and TensorFlow combinations, pushed into Docker or conda so every machine runs the exact same stack.
Performance work
Retracing analysis, mixed precision rollout, input pipeline tuning with tf.data, and benchmarking that reports step time honestly rather than averaged over the warmup.
TF1 to TF2 migration
Porting legacy tf.Session and tf.contrib code to eager execution and Keras, with tests that prove the output is the same before and after.
Serving and deployment
TensorFlow Serving, TF Lite conversion, quantization, and browser-side TensorFlow.js deployment, all tuned against the real workload instead of a toy.
Managed Tech
Framework upgrades rehearsed, GPU utilization monitored, model versions tracked, and an engineer available when a driver update or TF minor release breaks something overnight.

How it works

From first message to fixed

  1. Look Free estimate

    Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.

  2. Fix

    We fix the cause, not the symptom. If something is down, getting it back up comes first.

  3. Harden

    Backups, monitoring, updates and access, so the same problem can't quietly come back.

  4. Hand back or stay on

    You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.

Often found together

Running TensorFlow alongside something else?

We work on the whole stack, so the problem can be wherever it actually is.

Questions

TensorFlow questions

Something else? Call+1 737 727 7327

Should we switch from TensorFlow to PyTorch?

For new research work, usually yes. For a production stack built around TensorFlow Serving, TF Lite or TPUs, almost always no. We look at what your existing investment actually buys you, what the migration would cost and what you would gain, and only recommend the move when the arithmetic is clearly on its side.

Why does TensorFlow run out of GPU memory on the second run?

TensorFlow grabs most of the device memory on first use by default. A second notebook or script in the same process space finds only scraps left. Setting tf.config.experimental.set_memory_growth(True) on each GPU, or capping the per-process fraction, usually fixes it. We also check for batch sizes tuned on a bigger card than the one currently in use.

How do we resolve CUDA version mismatches?

Pin the TensorFlow build to a specific CUDA and cuDNN pair that match it exactly, package the stack in Docker or conda so every machine is identical, and refuse driver upgrades that are not tested first. We produce a documented environment image you can rebuild from scratch when needed.

Why does our tf.function retrace on every call?

The function is being called with inputs whose shapes or types change between invocations, or it contains a Python-level operation that TensorFlow cannot bake into the graph. Both are visible in the retracing warnings. The fix is to pass fixed-shape tensors or input signatures, and to move Python logic outside the traced function.

Can we still use TensorFlow for new projects in 2026?

Yes, especially for production inference, mobile deployment and TPU-based training. For purely research-driven work that will share code with the wider community, PyTorch is a less lonely choice. The two frameworks can also coexist: train in one, deploy in the other, if the trade-offs point that way.

How much does TensorFlow help cost?

An initial estimate is free. Send the error, the hardware, the TensorFlow version and a slice of the code, and we respond with a priced plan before any engineering starts.

Free estimate

TensorFlow giving you trouble?

Share the stack trace, the CUDA and TF versions and the hardware. A real engineer sends back a quote and a plan before any work begins.

  • No charge to find out what it will take
  • If it's urgent, say so

Replies come from a person. By sending, you authorize us to email you back. No marketing.