Skip to content

fix: let workspace pages download partial logs for unhealthy workspaces #13761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jul 3, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions site/src/api/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,15 +110,15 @@ export const getValidationErrorMessage = (error: unknown): string => {
return validationErrors.map((error) => error.detail).join("\n");
};

export const getErrorDetail = (error: unknown): string | undefined | null => {
export const getErrorDetail = (error: unknown): string | null => {
if (error instanceof Error) {
return "Please check the developer console for more details.";
}
if (isApiError(error)) {
return error.response.data.detail;
return error.response.data.detail ?? null;
}
if (isApiErrorResponse(error)) {
return error.detail;
return error.detail ?? null;
}
return null;
};
30 changes: 23 additions & 7 deletions site/src/modules/resources/AgentLogs/useAgentLogs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,38 @@ export type UseAgentLogsOptions = Readonly<{
enabled?: boolean;
}>;

/**
* Defines a custom hook that gives you all workspace agent logs for a given
* workspace.
*
* Depending on the status of the workspace, all logs may or may not be
* available.
*/
export function useAgentLogs(
options: UseAgentLogsOptions,
): readonly WorkspaceAgentLog[] | undefined {
const { workspaceId, agentId, agentLifeCycleState, enabled = true } = options;

const queryClient = useQueryClient();
const queryOptions = agentLogs(workspaceId, agentId);
const query = useQuery({
...queryOptions,
enabled,
});
const logs = query.data;
const query = useQuery({ ...queryOptions, enabled });

// One pitfall with the current approach: the enabled property does NOT
// prevent the useQuery call above from eventually having data. All it does
// is prevent it from getting data on its own. Let's say a different useQuery
// call elsewhere in the app has the same query key and is enabled. When it
// gets data back from the server, the useQuery call here will re-render with
// that same new data, even though this state is "disabled". This can EASILY
// cause bugs.
const logs = enabled ? query.data : undefined;

const lastQueriedLogId = useRef(0);
useEffect(() => {
if (logs && lastQueriedLogId.current === 0) {
lastQueriedLogId.current = logs[logs.length - 1].id;
const lastLog = logs?.at(-1);
const canSetLogId = lastLog !== undefined && lastQueriedLogId.current === 0;

if (canSetLogId) {
lastQueriedLogId.current = lastLog.id;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously: we were checking that the array existed, but not that it actually had data

Really do think that we need to bite the bullet and get TypeScript's noUncheckedIndexedAccess compiler option turned on, because it would've caught this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good one and would require a ticket and PR by itself.

}
}, [logs]);

Expand Down
244 changes: 187 additions & 57 deletions site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx
Original file line number Diff line number Diff line change
@@ -1,17 +1,25 @@
import { useTheme, type Interpolation, type Theme } from "@emotion/react";
import Skeleton from "@mui/material/Skeleton";
import ErrorIcon from "@mui/icons-material/ErrorOutline";

Check failure on line 3 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

`@mui/icons-material/ErrorOutline` import should occur before import of `@mui/material/Skeleton`

Check failure on line 3 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

`@mui/icons-material/ErrorOutline` import should occur before import of `@mui/material/Skeleton`
import { saveAs } from "file-saver";
import JSZip from "jszip";
import { useMemo, useState, type FC } from "react";
import { useQueries, useQuery } from "react-query";
import { type FC, useMemo, useState, useRef, useEffect } from "react";
import { UseQueryOptions, useQueries, useQuery } from "react-query";

Check failure on line 7 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

Import "UseQueryOptions" is only used as types

Check failure on line 7 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

Import "UseQueryOptions" is only used as types
import { agentLogs, buildLogs } from "api/queries/workspaces";
import type { Workspace, WorkspaceAgent } from "api/typesGenerated";
import type {
Workspace,
WorkspaceAgent,
WorkspaceAgentLog,
} from "api/typesGenerated";
import {
ConfirmDialog,
type ConfirmDialogProps,
} from "components/Dialogs/ConfirmDialog/ConfirmDialog";
import { displayError } from "components/GlobalSnackbar/utils";
import { Stack } from "components/Stack/Stack";
import { ErrorAlert } from "components/Alert/ErrorAlert";

Check failure on line 20 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

`components/Alert/ErrorAlert` import should occur before import of `components/Dialogs/ConfirmDialog/ConfirmDialog`

Check failure on line 20 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

`components/Alert/ErrorAlert` import should occur before import of `components/Dialogs/ConfirmDialog/ConfirmDialog`

const BLOB_SIZE_UNITS = ["B", "KB", "MB", "GB", "TB"] as const;

type DownloadLogsDialogProps = Pick<
ConfirmDialogProps,
Expand All @@ -28,70 +36,124 @@

export const DownloadLogsDialog: FC<DownloadLogsDialogProps> = ({
workspace,
open,
onClose,
download = saveAs,
...dialogProps
}) => {
const theme = useTheme();
const agents = selectAgents(workspace);
const agentLogResults = useQueries({
queries: agents.map((a) => ({
...agentLogs(workspace.id, a.id),
enabled: dialogProps.open,
})),
});

const buildLogsQuery = useQuery({
...buildLogs(workspace),
enabled: dialogProps.open,
enabled: open,
});
const downloadableFiles: DownloadableFile[] = useMemo(() => {
const files: DownloadableFile[] = [
{
name: `${workspace.name}-build-logs.txt`,
blob: buildLogsQuery.data
? new Blob([buildLogsQuery.data.map((l) => l.output).join("\n")], {
type: "text/plain",
})
: undefined,
},
const buildLogsFile = useMemo<DownloadableFile>(() => {
return {
name: `${workspace.name}-build-logs.txt`,
blob: buildLogsQuery.data
? new Blob([buildLogsQuery.data.map((l) => l.output).join("\n")], {
type: "text/plain",
})
: undefined,
};
}, [workspace.name, buildLogsQuery.data]);

// This is clunky, but we have to memoize in two steps to make sure that we
// don't accidentally break the memo cache every render. We can't tuck
// everything into a single memo call, because we need to set up React Query
// state between processing the agents, but we can't violate rules of hooks by
// putting hooks inside of hooks
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, but I feel it is an over-optimization. If it is fast enough and does not impact UI usage, I don't see why we should complicate things.

Copy link
Member Author

@Parkreiner Parkreiner Jul 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was considering that, but didn't really have time to test out how long it would take to keep generating blobs every render

To be clear, we basically have two choices:

  • Keep the clunky but functional memoization
  • Remove the memoization altogether

The original memoization has worse performance than not having memoization at all, because we're still recalculating the blobs every render, but React has to compare dependencies and decide if it should re-calculate the value

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it was my fault. I usually use "useMemo" to compute a variable value but it would probably be better to be a function. I would rather remove memorization altogether to make things simple.

type AgentInfo = Readonly<{
agents: readonly WorkspaceAgent[];
logOptionsArray: readonly UseQueryOptions<readonly WorkspaceAgentLog[]>[];
}>;

const { agents, logOptionsArray } = useMemo<AgentInfo>(() => {
const allAgents = workspace.latest_build.resources.flatMap(
(resource) => resource.agents ?? [],
);

// Can't use the "new Set()" trick because we're not dealing with primitives
const uniqueAgents = [
...new Map(allAgents.map((agent) => [agent.id, agent])).values(),
];

return {
agents: uniqueAgents,
logOptionsArray: uniqueAgents.map((agent) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to calculate the query options here and not mount it and pass it in the query? I think it is kinda far from what is used and I took some time to link logOptionsArray to agentQueryOptions

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this was an over-optimization on my part. I basically figured that since the query objects shouldn't change unless the agents do, it was safe to throw them all together

return {
...agentLogs(workspace.id, agent.id),
enabled: open,
};
}),
};
}, [workspace, open]);

const agentLogQueries = useQueries({ queries: logOptionsArray });
const allFiles = useMemo<readonly DownloadableFile[]>(() => {
const files: DownloadableFile[] = [buildLogsFile];

agents.forEach((a, i) => {
const name = `${a.name}-logs.txt`;
const logs = agentLogResults[i].data;
const txt = logs?.map((l) => l.output).join("\n");
const txt = agentLogQueries[i]?.data?.map((l) => l.output).join("\n");

let blob: Blob | undefined;
if (txt) {
blob = new Blob([txt], { type: "text/plain" });
}

files.push({ name, blob });
});

return files;
}, [agentLogResults, agents, buildLogsQuery.data, workspace.name]);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a problem with the previous logic: because agents was getting redefined every render, we were also invalidating the useMemo dependency array every render

const isLoadingFiles = downloadableFiles.some((f) => f.blob === undefined);
}, [agentLogQueries, agents, buildLogsFile]);

const [isDownloading, setIsDownloading] = useState(false);
const isWorkspaceHealthy = workspace.health.healthy;
const isLoadingFiles = allFiles.some((f) => f.blob === undefined);

const resetDownloadStateIdRef = useRef<number | undefined>(undefined);
useEffect(() => {
const clearTimeoutOnUnmount = () => {
window.clearTimeout(resetDownloadStateIdRef.current);
};

return clearTimeoutOnUnmount;
}, []);

return (
<ConfirmDialog
{...dialogProps}
open={open}
onClose={onClose}
hideCancel={false}
title="Download logs"
confirmText="Download"
disabled={isLoadingFiles}
confirmLoading={isDownloading}
confirmText={
<>
Download
{!isWorkspaceHealthy && <> {isLoadingFiles ? "partial" : "all"}</>}
</>
}
disabled={
isDownloading ||
// If a workspace isn't healthy, let the user download as many logs as
// they can. Otherwise, wait for everything to come in
(isWorkspaceHealthy && isLoadingFiles)
}
onConfirm={async () => {
setIsDownloading(true);
const zip = new JSZip();
allFiles.forEach((f) => {
if (f.blob) {
zip.file(f.name, f.blob);
}
});
Comment on lines +119 to +125
Copy link
Member Author

@Parkreiner Parkreiner Jul 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this out of the try block, because it should be the part that shouldn't be able to fail


try {
setIsDownloading(true);
const zip = new JSZip();
downloadableFiles.forEach((f) => {
if (f.blob) {
zip.file(f.name, f.blob);
}
});
const content = await zip.generateAsync({ type: "blob" });
download(content, `${workspace.name}-logs.zip`);
dialogProps.onClose();
setTimeout(() => {
onClose();

resetDownloadStateIdRef.current = window.setTimeout(() => {
setIsDownloading(false);
}, theme.transitions.duration.leavingScreen);
} catch (error) {
Expand All @@ -106,18 +168,18 @@
Downloading logs will create a zip file containing all logs from all
jobs in this workspace. This may take a while.
</p>

{!isWorkspaceHealthy && isLoadingFiles && (
<ErrorAlert error="Your workspace is not healthy. Some logs may be unavailable." />
)}

<ul css={styles.list}>
{downloadableFiles.map((f) => (
<li key={f.name} css={styles.listItem}>
<span css={styles.listItemPrimary}>{f.name}</span>
<span css={styles.listItemSecondary}>
{f.blob ? (
humanBlobSize(f.blob.size)
) : (
<Skeleton variant="text" width={48} height={12} />
)}
</span>
</li>
{allFiles.map((f) => (
<DownloadingItem
key={f.name}
file={f}
giveUpTimeMs={isWorkspaceHealthy ? undefined : 5_000}
/>
))}
</ul>
</Stack>
Expand All @@ -126,20 +188,70 @@
);
};

type DownloadingItemProps = Readonly<{
// A value of undefined indicates that the component will wait forever
giveUpTimeMs?: number;
file: DownloadableFile;
}>;

const DownloadingItem: FC<DownloadingItemProps> = ({ file, giveUpTimeMs }) => {
const theme = useTheme();
const [isWaiting, setIsWaiting] = useState(true);
useEffect(() => {
if (giveUpTimeMs === undefined || file.blob !== undefined) {
setIsWaiting(true);
return;
}

const timeoutId = window.setTimeout(
() => setIsWaiting(false),
giveUpTimeMs,
);

return () => window.clearTimeout(timeoutId);
}, [giveUpTimeMs, file]);

return (
<li css={styles.listItem}>
<span
css={[
styles.listItemPrimary,
!isWaiting && { color: theme.palette.text.disabled },
]}
>
{file.name}
</span>

<span css={styles.listItemSecondary}>
{file.blob ? (
humanBlobSize(file.blob.size)
) : isWaiting ? (
<Skeleton variant="text" width={48} height={12} />
) : (
<div css={styles.notAvailableText}>
<span aria-hidden>
<ErrorIcon fontSize={"inherit"} />

Check failure on line 233 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

Curly braces are unnecessary here

Check failure on line 233 in site/src/pages/WorkspacePage/WorkspaceActions/DownloadLogsDialog.tsx

View workflow job for this annotation

GitHub Actions / lint

Curly braces are unnecessary here
</span>

<p>N/A</p>
</div>
)}
</span>
</li>
);
};

function humanBlobSize(size: number) {
const units = ["B", "KB", "MB", "GB", "TB"];
let i = 0;
while (size > 1024 && i < units.length) {
while (size > 1024 && i < BLOB_SIZE_UNITS.length) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the BLOB_SIZE_UNITS is only used here, I would not move it to leave outside of the function where it is used.

size /= 1024;
i++;
}
return `${size.toFixed(2)} ${units[i]}`;
}

function selectAgents(workspace: Workspace): WorkspaceAgent[] {
return workspace.latest_build.resources
.flatMap((r) => r.agents)
.filter((a) => a !== undefined) as WorkspaceAgent[];
// The while condition can break if we accidentally exceed the bounds of the
// array. Have to be extra sure we have a unit at the very end.
const finalUnit = BLOB_SIZE_UNITS[i] ?? BLOB_SIZE_UNITS.at(-1) ?? "TB";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't get this 😞

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry – just updated the wording on the comment for more clarity!

return `${size.toFixed(2)} ${finalUnit}`;
}

const styles = {
Expand All @@ -163,4 +275,22 @@
listItemSecondary: {
fontSize: 14,
},

notAvailableText: (theme) => ({
display: "flex",
flexFlow: "row nowrap",
alignItems: "center",
columnGap: "4px",

"& > span": {
maxHeight: "fit-content",
display: "flex",
alignItems: "center",
color: theme.palette.error.light,
},

"& > p": {
opacity: "80%",
},
}),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been stuck in the pre-Emotion, Backstage-specific version of MUI for the past few months, so please let me know if anything looks funky

} satisfies Record<string, Interpolation<Theme>>;
Loading