Skip to content

Include host config, network and copied files in the container reuse hash - #1624

Draft
ankit1324 wants to merge 1 commit into
testcontainers:mainfrom
ankit1324:fix/reuse-hash-host-config
Draft

ankit1324 wants to merge 1 commit into
testcontainers:mainfrom
ankit1324:fix/reuse-hash-host-config

Conversation

@ankit1324

Copy link
Copy Markdown

Summary

withReuse() hashed only createOpts (reuseOrStartContainer), but the container is created with
{ ...createOpts, HostConfig: hostConfig }, attached to networkMode with networkAliases after create, and
gets the copied files and content after create. Two builders that differed in tmpfs, privileged mode, extra hosts,
port bindings, bind mounts, network aliases or copied content got the same hash, so the second one was handed the
first one's container.

The hash input is now built in one place (createContainerHash) and covers createOpts, hostConfig, the
network mode and aliases, the copy inputs and copyToContainerOptions. It is still stored in the single
org.testcontainers.container-hash label.

Copied files and directories are hashed by content (target, mode, file mode and bytes; directories walked
recursively in sorted order), following testcontainers-java's
hashCopiedFiles
and how it feeds into the create-command hash, which also covers the host config and network settings
(L386-L402).
Unlike Java, copy entries keep their insertion order (a later copy can overwrite an earlier one) and directory
entries include their relative path. String and Buffer content is hashed directly. A Readable content stream or
a tar archive can't be hashed without consuming it, so only its target (and mode) is included. The docs now say so,
and that the network is part of the configuration.

The host.testcontainers.internal extra host that start() adds when host ports are exposed is left out of the
hash, because the port forwarder's IP changes every session and including it would create a new reusable
container per session. This matches .NET's
JsonIgnoreRuntimeExtraHosts.
Extra hosts set with withExtraHosts are hashed. The trade-off is the same as on main and in .NET: a reused
container keeps the extra host it was created with (none, or an old forwarder IP) and is not connected to the
current port forwarder network, because reuseContainer doesn't reconnect networks.

Settings that don't change the created container (auto-remove, wait strategy, startup timeout, pull policy, log
consumer) stay out of the hash, and the version label is untouched (#1552).

Known limitations:

  • Object keys are hashed in insertion order, as before for createOpts. Builders that set the same env, labels or
    tmpfs in a different order get different hashes. That starts an extra container; it never reuses the wrong one.
  • Every start() with reuse enabled reads and hashes all copied files and walks copied directories, even when an
    existing container is reused (Java does the same).

Verification

  • npx vitest run packages/testcontainers/src/generic-container/generic-container-reuse.test.ts
    • before the fix: 8 failed | 14 passed (22). Tmpfs, privileged mode, extra host, bind mount, copied content,
      network aliases, copied file and copied directory all reused the first container
      (expected '<id>' not to be '<id>').
    • after: 22 passed (22).
    • guards that pass before and after: identical builders (host config and copied content, stream content) still
      reuse, and a container created before TestContainers.exposeHostPorts() keeps its hash after it. The last one
      fails if the port forwarder entry is hashed.
  • npx vitest run packages/testcontainers/src/generic-container packages/testcontainers/src/port-forwarder packages/testcontainers/src/network packages/testcontainers/src/common:
    18 files, 142 tests, 1 skipped. Tests in generic-container-dockerfile.test.ts ("Failed to build image") and
    "should bind to specified host port with a different protocol" failed intermittently while another Docker-heavy
    test run shared the daemon, and passed on rerun. They don't use reuse.
  • common/hash.test.ts has a new unit test for hashFile.
  • npm run format, npm run lint, npm run knip, npm run check-compiles: clean.
  • Not run: the Kafka reuse tests (they pull Kafka images). Kafka's beforeContainerCreated only writes
    deterministic config, so they are expected to still reuse.

Not breaking

No API changes. Existing reusable containers get a new hash once, so the first run after upgrading starts a new
container instead of reusing an old one. This already happens on every release because the hash includes the
library version label.

Identical configurations still reuse, with these behaviour changes:

  • The network name is now part of the hash. A container started on a newly created Network (which gets a
    random name) is no longer reused across runs, and AzureServiceBusContainer, which creates a new Network on
    every start(), no longer reuses. Before, these were reused, but the reused container was never connected to
    the new network. Containers started with withNetworkMode and a fixed network name are unaffected.
  • On the reuse path, a missing withCopyFilesToContainer source now fails before the container is created,
    instead of after create when the files are copied.

Closes #1522

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include host config and copied files in the container reuse hash

1 participant