Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions api/stovepipe/proto/stovepipe.proto
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,44 @@ message GetQueueStatusResponse {
QueueEnablementStatus enablement = 2;
}

// IssueSummary is the current materialized view of a validation issue.
message IssueSummary {
// Logical validation task.
string task_id = 1;
// Validation unit; empty identifies a task-wide issue.
string unit_id = 2;
// Materialized status. ListOpenIssues returns only "open".
string status = 3;
// Build where the current or most recent opening was first observed.
string first_seen_build_id = 4;
// Source request of the latest status transition.
string request_id = 5;
// Source build of the latest status transition.
string build_id = 6;
// Original transition timestamp in Unix milliseconds.
int64 changed_at_ms = 7;
}

// ListOpenIssuesRequest selects current open issues within one queue.
message ListOpenIssuesRequest {
// Required configured queue.
string queue = 1;
// Maximum issues to return. Zero selects 50; otherwise 1 through 200.
int32 page_size = 2;
// Opaque continuation bound to the queue and task/unit ordering.
string page_token = 3;
}

// ListOpenIssuesResponse is not a snapshot across calls.
message ListOpenIssuesResponse {
// Open summaries in ascending bytewise task/unit order.
// The inspection budget can leave this partial or empty even with a continuation.
repeated IssueSummary issues = 1;
// Continues after the last examined open-issue row, including skipped references.
// Empty means no further open-issue row was observed.
string next_page_token = 2;
}

// Stovepipe provides the Stovepipe API.
service Stovepipe {
// Ping returns a response indicating the service is alive
Expand All @@ -309,4 +347,6 @@ service Stovepipe {
rpc List(ListRequest) returns (ListResponse) {}
// GetQueueStatus returns current enablement or enablement for a supplied commit.
rpc GetQueueStatus(GetQueueStatusRequest) returns (GetQueueStatusResponse) {}
// ListOpenIssues returns current open issues validated against their summaries.
rpc ListOpenIssues(ListOpenIssuesRequest) returns (ListOpenIssuesResponse) {}
}
297 changes: 271 additions & 26 deletions api/stovepipe/protopb/stovepipe.pb.go

Large diffs are not rendered by default.

235 changes: 147 additions & 88 deletions api/stovepipe/protopb/stovepipe.pb.yarpc.go

Large diffs are not rendered by default.

40 changes: 40 additions & 0 deletions api/stovepipe/protopb/stovepipe_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 21 additions & 0 deletions doc/rfc/stovepipe/open-issues-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Open issues API

`ListOpenIssues` returns current open issue summaries for one configured queue in ascending bytewise task/unit order. An empty unit identifies a task-wide issue. The response exposes summary data without internal storage versions.

## Bounded pagination

The request selects a queue, page size, and opaque continuation token. Page size defaults to 50 and is limited to 200. It bounds the number of matching issues returned. The controller scans until the page fills, the lookup range is exhausted, or 500 lookup entries have been validated. Each storage batch reads one additional entry to detect whether the range continues; this lookahead entry is not validated or consumed by that batch.

Continuations are bound to the selected queue and advance after the last examined task/unit key. Page size may change between calls. Malformed tokens, mismatched queues, unconfigured queues, and invalid page sizes are rejected.

A page can contain fewer issues than requested, including zero, while still carrying a continuation when the inspection budget is reached. Clients must use the continuation token rather than response length to decide whether to keep reading. Pagination is not a snapshot: summaries and lookup membership can change during or between calls.

## Summary validation

The controller scans bounded ranges from the open-issue lookup and point-reads the corresponding summaries. It returns a summary only when its status is open and its opening matches the lookup reference. References to closed issues or previous openings are skipped. Skipped references still advance the cursor and count against the inspection budget.

Summary-read failures fail the request rather than returning a partial page. A missing summary is also an error: summaries retain closed rows, and lookup writers persist the summary first. A lookup that returns a different task/unit identity from its summary is an infrastructure inconsistency.

The API is read-only. It does not repair open-issue rows, scan operational issue state, perform database joins, or require secondary indexes. The issue-log materializer maintains the lookup during normal processing and recovery.

An empty result does not establish build success or complete issue coverage. Historical rows without a materialized lookup are not backfilled by reads. Project-specific filtering, issue history, repair details, and the frontend are outside this API.
1 change: 1 addition & 0 deletions service/stovepipe/server/handler/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ go_library(
go_test(
name = "go_default_test",
srcs = [
"list_open_issues_test.go",
"list_test.go",
"queue_status_test.go",
"server_test.go",
Expand Down
64 changes: 64 additions & 0 deletions service/stovepipe/server/handler/list_open_issues_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package handler

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
pb "github.com/uber/submitqueue/api/stovepipe/protopb"
"github.com/uber/submitqueue/stovepipe/controller"
controllermock "github.com/uber/submitqueue/stovepipe/controller/mock"
"github.com/uber/submitqueue/stovepipe/entity"
"go.uber.org/mock/gomock"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)

func TestListOpenIssues(t *testing.T) {
for _, tt := range []struct {
name string
err error
code codes.Code
}{
{name: "success"},
{name: "invalid input", err: controller.ErrInvalidRequest, code: codes.InvalidArgument},
{name: "canceled", err: context.Canceled, code: codes.Canceled},
{name: "deadline", err: context.DeadlineExceeded, code: codes.DeadlineExceeded},
{name: "storage failure", err: assert.AnError, code: codes.Unknown},
} {
t.Run(tt.name, func(t *testing.T) {
c := controllermock.NewMockListOpenIssuesController(gomock.NewController(t))
result := entity.ListOpenIssuesResult{Issues: []entity.IssueSummary{{TaskID: "task", Status: entity.IssueStatusOpen, FirstSeenBuildID: "opening"}}, NextPageToken: "next"}
c.EXPECT().ListOpenIssues(gomock.Any(), entity.ListOpenIssuesRequest{Queue: "queue", PageSize: 10, PageToken: "token"}).Return(result, tt.err)
server := &StovepipeServer{openIssuesController: c}
got, err := server.ListOpenIssues(context.Background(), &pb.ListOpenIssuesRequest{Queue: "queue", PageSize: 10, PageToken: "token"})
if tt.err != nil {
require.Equal(t, tt.code, status.Code(err))
require.Nil(t, got)
if tt.err == assert.AnError {
require.ErrorIs(t, err, tt.err)
}
} else {
require.NoError(t, err)
require.Equal(t, "task", got.GetIssues()[0].GetTaskId())
require.Equal(t, "open", got.GetIssues()[0].GetStatus())
require.Equal(t, "next", got.GetNextPageToken())
}
})
}
}
2 changes: 1 addition & 1 deletion service/stovepipe/server/handler/list_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ func TestList(t *testing.T) {
}, req)
return tt.result, tt.err
}}
srv := NewStovepipeServer(nil, nil, nil, nil, fake, nil)
srv := NewStovepipeServer(nil, nil, nil, nil, fake, nil, nil)
response, err := srv.List(ctx, &pb.ListRequest{
Queue: "queue", PageSize: 25, PageToken: "token",
AcceptedLowerBound: &pb.ListRequest_AcceptedAtOrAfterMs{},
Expand Down
2 changes: 1 addition & 1 deletion service/stovepipe/server/handler/queue_status_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ func TestGetQueueStatus(t *testing.T) {
require.Equal(t, entity.GetQueueStatusRequest{Queue: "repo/main", ChangeURI: "C"}, req)
return entity.GetQueueStatusResult{Queue: "repo/main", Enablement: entity.QueueState{Version: 1, State: entity.QueueEnablementDisabled}}, wantErr
}}
server := NewStovepipeServer(nil, nil, nil, nil, nil, fake)
server := NewStovepipeServer(nil, nil, nil, nil, nil, fake, nil)
response, err := server.GetQueueStatus(context.Background(), &pb.GetQueueStatusRequest{Queue: "repo/main", ChangeUri: "C"})
if wantErr != nil {
require.Equal(t, tt.code, status.Code(err))
Expand Down
18 changes: 18 additions & 0 deletions service/stovepipe/server/handler/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ type StovepipeServer struct {
requestHistoryController controller.RequestHistoryController
projectStatusController controller.ProjectStatusController
listController controller.ListController
openIssuesController controller.ListOpenIssuesController
queueStatusController controller.GetQueueStatusController
}

Expand All @@ -48,13 +49,15 @@ func NewStovepipeServer(
projectStatusController controller.ProjectStatusController,
listController controller.ListController,
queueStatusController controller.GetQueueStatusController,
openIssuesController controller.ListOpenIssuesController,
) *StovepipeServer {
return &StovepipeServer{
pingController: pingController,
ingestController: ingestController,
requestHistoryController: requestHistoryController,
projectStatusController: projectStatusController,
listController: listController,
openIssuesController: openIssuesController,
queueStatusController: queueStatusController,
}
}
Expand Down Expand Up @@ -143,3 +146,18 @@ func (s *StovepipeServer) GetQueueStatus(ctx context.Context, req *pb.GetQueueSt
}
return mapper.GetQueueStatusResultToProto(result), nil
}

func (s *StovepipeServer) ListOpenIssues(ctx context.Context, req *pb.ListOpenIssuesRequest) (*pb.ListOpenIssuesResponse, error) {
result, err := s.openIssuesController.ListOpenIssues(ctx, mapper.ProtoToListOpenIssuesRequest(req))
if err != nil {
switch {
case controller.IsInvalidRequest(err):
return nil, status.Error(codes.InvalidArgument, err.Error())
case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
return nil, status.FromContextError(err).Err()
default:
return nil, err
}
}
return mapper.ListOpenIssuesResultToProto(result), nil
}
6 changes: 5 additions & 1 deletion service/stovepipe/server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,11 @@ func run() error {
projectStatusController := controller.NewProjectStatusController(logger.Sugar(), scope, storageFty)
listController := controller.NewListController(logger.Sugar(), scope, storageFty, tenants)
queueStatusController := controller.NewGetQueueStatusController(scope, storageFty, sourceControl)
srv := handler.NewStovepipeServer(pingController, ingestController, requestHistoryController, projectStatusController, listController, queueStatusController)
openIssuesController := controller.NewListOpenIssuesController(logger.Sugar(), scope, storageFty, tenants)
srv := handler.NewStovepipeServer(
pingController, ingestController, requestHistoryController, projectStatusController,
listController, queueStatusController, openIssuesController,
)
pb.RegisterStovepipeServer(grpcServer, srv)

// Register reflection service for debugging with grpcurl
Expand Down
2 changes: 2 additions & 0 deletions service/stovepipe/server/mapper/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go_library(
srcs = [
"ingest.go",
"list.go",
"list_open_issues.go",
"project_status.go",
"queue_status.go",
"request_history.go",
Expand All @@ -21,6 +22,7 @@ go_test(
name = "go_default_test",
srcs = [
"ingest_test.go",
"list_open_issues_test.go",
"list_test.go",
"project_status_test.go",
"queue_status_test.go",
Expand Down
42 changes: 42 additions & 0 deletions service/stovepipe/server/mapper/list_open_issues.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package mapper

import (
pb "github.com/uber/submitqueue/api/stovepipe/protopb"
"github.com/uber/submitqueue/stovepipe/entity"
)

// ProtoToListOpenIssuesRequest leaves defaults and validation to the controller.
func ProtoToListOpenIssuesRequest(req *pb.ListOpenIssuesRequest) entity.ListOpenIssuesRequest {
return entity.ListOpenIssuesRequest{Queue: req.GetQueue(), PageSize: req.GetPageSize(), PageToken: req.GetPageToken()}
}

// ListOpenIssuesResultToProto preserves summary order and the continuation token.
func ListOpenIssuesResultToProto(result entity.ListOpenIssuesResult) *pb.ListOpenIssuesResponse {
issues := make([]*pb.IssueSummary, len(result.Issues))
for i, summary := range result.Issues {
issues[i] = &pb.IssueSummary{
TaskId: summary.TaskID,
UnitId: summary.UnitID,
Status: string(summary.Status),
FirstSeenBuildId: summary.FirstSeenBuildID,
RequestId: summary.RequestID,
BuildId: summary.BuildID,
ChangedAtMs: summary.ChangedAtMs,
}
}
return &pb.ListOpenIssuesResponse{Issues: issues, NextPageToken: result.NextPageToken}
}
Loading
Loading