Repository navigation
feat: Kafka agent - #1028
feat: Kafka agent#1028labrenbe wants to merge 25 commits into
Conversation
|
|
||
| [workspace.dependencies] | ||
| stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "stackable-operator-0.118.0", features = ["crds", "webhook"] } | ||
| stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", branch = "feat/platform-access", features = ["crds", "webhook"] } |
There was a problem hiding this comment.
TODO: revert to a tag when ready.
| .with_labels(recommended_labels_for_agent_resources(cluster)) | ||
| .build(), | ||
| ) | ||
| .image_pull_secrets_from_product_image(&cluster.image) |
There was a problem hiding this comment.
Assuming that the agent image can use the same credentials as the product image is fine as long as they come from the same registry. If they don't, there should be an optional field platformAccess.image.pullSecrets that allows the user to specify these independently; when unset, the agent falls back to spec.image.pullSecrets as it currently does.
There was a problem hiding this comment.
This is currently following the precedence of opa-operator. If that pull secret doesn't work then objectOverrides can still be used to override it. I think it's worth a discussion how much customization we want to allow for the agent without overrides.
9b6f2a7 to
146602d
Compare
| ) | ||
| .expect("the volume name is statically defined and must not collide"); | ||
| container_builder | ||
| .add_volume_mount(&*DISCOVERY_VOLUME_NAME, DISCOVERY_DIR) |
There was a problem hiding this comment.
Just a question if this is intended behavior: if the address in the discovery dir is an external one (external ListenerClass), is it intended that the agent connects from outside using that external path?
| agent_config: &ValidatedAgentConfig, | ||
| ) -> Deployment { | ||
| let resource_names = cluster.agent_resource_names(); | ||
| let mut cb_agent = new_container_builder(&AGENT_CONTAINER_NAME); |
There was a problem hiding this comment.
| let mut cb_agent = new_container_builder(&AGENT_CONTAINER_NAME); | |
| let mut container_builder = new_container_builder(&AGENT_CONTAINER_NAME); |
So it's consistent with the pod_builder below
There was a problem hiding this comment.
The thing is there might be multiple containers, but a single pod. I guess we would need container_builder_agent in this case, which is quite a mouthful ;)
There was a problem hiding this comment.
I would still prefer it over cb_agent :D would also be in line with the code style guidelines
There was a problem hiding this comment.
container_builder_agent works for me ;)
| ensure!( | ||
| metadata_manager == crate::crd::MetadataManager::KRaft, | ||
| ManagementAccessRequiresKraftSnafu | ||
| ); |
There was a problem hiding this comment.
This is pretty arbitrary. Can we document / comment why?
There was a problem hiding this comment.
I decided to focus on KRaft for the first version of the admin client as supporting clusters with zookeeper would add extra complexity. I will document that this is a current limitation of the first version of the kafka-agent. Similarly we decided not to support Kerberos for version 1 in stackabletech/issues#888.
Description
Part of stackabletech/issues#888.
Depends on decisions
Depends on PRs
Deploys an agent alongside Kafka clusters which is an independent controller that reconciles and manages internal resources like
KafkaTopic.This PR only adds an empty agent shell that doesn't do anything yet.
Definition of Done Checklist
Author
Reviewer
Acceptance
type/deprecationlabel & add to the deprecation scheduletype/experimentallabel & add to the experimental features tracker