Skip to content

Handle pod logs cut mid-character by limitBytes - #2761

Open
carterpewpew wants to merge 1 commit into
kubernetes-client:masterfrom
carterpewpew:limit-bytes-truncated-utf8
Open

carterpewpew wants to merge 1 commit into
kubernetes-client:masterfrom
carterpewpew:limit-bytes-truncated-utf8

Conversation

@carterpewpew

Copy link
Copy Markdown

What type of PR is this?

/kind bug

What this PR does / why we need it:

read_namespaced_pod_log(limit_bytes=N) returns the first N bytes of the log, and the API server does not align that cut to a character boundary. When the cut lands inside a multi-byte UTF-8 character, response_deserialize() raises UnicodeDecodeError and the whole log is lost.

This still happens on 1.37. The decode moved from rest.py (where the trace in the issue points) to ApiClient.response_deserialize(). With a log body of café € cut one byte short, 37.0.1 fails with 'utf-8' codec can't decode bytes in position 6-7: unexpected end of data.

When the strict decode fails, the body is decoded again with an incremental decoder and final=False. That leaves out only an incomplete sequence at the very end, so the example above returns café . Any other invalid byte still raises UnicodeDecodeError, as before. Valid responses take the same strict path as today. bytes, file and _preload_content=False responses are not affected.

api_client.py is generated, so the same change is kept in scripts/api_client_patch.diff and scripts/api_client_asyncio_patch.diff, and update-client.sh / update-client-asyncio.sh apply them after generation, like the keepalive and client-go retry patches.

Which issue(s) this PR fixes:

Fixes #2756

Special notes for your reviewer:

The new tests go through CoreV1Api.read_namespaced_pod_log for both clients: the sync one with a mocked pool manager, the asyncio one against the aiohttp test server already in test_generated_api.py. The truncation tests fail on master with the error above. The invalid-byte tests fail if the decode is relaxed to errors="ignore".

Does this PR introduce a user-facing change?

Reading pod logs with `limit_bytes` no longer raises `UnicodeDecodeError` when the limit cuts a multi-byte UTF-8 character; the incomplete trailing character is dropped instead.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


This PR was made with AI assistance, but every change has been reviewed line by line.

read_namespaced_pod_log(limit_bytes=N) returns the first N bytes of the
log, and the API server does not align that cut to a character boundary.
When it falls inside a multi-byte UTF-8 character, response_deserialize()
raised UnicodeDecodeError and the caller lost the whole log.

When the strict decode fails, decode again with an incremental decoder
and final=False. That leaves out only an incomplete sequence at the very
end and still raises for any other invalid byte, so valid responses and
genuinely bad data behave as before.

The sync and asyncio api_client.py files are generated, so the change is
also kept in scripts/api_client_patch.diff and
scripts/api_client_asyncio_patch.diff, which the update scripts apply.

Signed-off-by: Jathavedhan M <[email protected]>
@kubernetes-prow kubernetes-prow Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. labels Oct 26, 2026
@kubernetes-prow
kubernetes-prow Bot requested review from roycaihw and yliaog October 26, 2026 06:42
@kubernetes-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: carterpewpew
Once this PR has been reviewed and has the lgtm label, please assign roycaihw for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Oct 26, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

Welcome @carterpewpew!

It looks like this is your first PR to kubernetes-client/python 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-client/python has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@kubernetes-prow kubernetes-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug Report: UnicodeDecodeError when limit_bytes truncates UTF-8 mid-character

1 participant