Skip to content

Progress notifications? #461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
olalonde opened this issue May 7, 2025 · 0 comments
Open

Progress notifications? #461

olalonde opened this issue May 7, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@olalonde
Copy link

olalonde commented May 7, 2025

I have a mcp server with a tool that can take several minutes to complete... Right now mcp clients usually consider it has timed out but it would be good if I was either 1) able to report progress on the task to avoid getting timed out 2) able to immediately return a message indicating the result will be returned later as a notification. Not sure which approach is recommended for that type of tool.

server.tool(
  "askHuman",
  {
    question: z.string().describe("The question to ask a human worker"),
    reward: z
      .string()
      .default("0.05")
      .describe("The reward amount in USD (default: $0.05)"),
    title: z.string().optional().describe("Title for the HIT (optional)"),
    description: z
      .string()
      .optional()
      .describe("Description for the HIT (optional)"),
    hitValiditySeconds: z
      .number()
      .default(3600)
      .describe("Time until the HIT expires in seconds (default: 1 hour)")
  },
  async ({
    question,
    reward,
    title,
    description,
    hitValiditySeconds,
  }) => {
    try {
      // Create HIT parameters
      // For GitHub Pages, use the direct HTML page URL
      // Default to local server if GITHUB_PAGES_URL is not set
      let formUrl;

      // Always use the GitHub Pages URL
      formUrl = new URL(FORM_SERVER_URL);

      // Add question and callback parameters
      formUrl.searchParams.append("question", encodeURIComponent(question));

      // If a callback URL is provided, add it to the form URL
      if (process.env.CALLBACK_URL) {
        formUrl.searchParams.append("callbackUrl", process.env.CALLBACK_URL);
      }

      const params = {
        Title: title || "Answer a question from an AI assistant",
        Description:
          description ||
          "Please provide your human perspective on this question",
        Question: `
          <ExternalQuestion xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2006-07-14/ExternalQuestion.xsd">
            <ExternalURL>${formUrl.toString()}</ExternalURL>
            <FrameHeight>600</FrameHeight>
          </ExternalQuestion>
        `,
        Reward: reward,
        MaxAssignments: 1,
        AssignmentDurationInSeconds: hitValiditySeconds,
        LifetimeInSeconds: hitValiditySeconds,
        AutoApprovalDelayInSeconds: 86400, // Auto-approve after 24 hours
      };

      // Create the HIT
      const createResult = await mturkClient.send(new CreateHITCommand(params));
      const hitId = createResult.HIT?.HITId;

      if (!hitId) {
        throw new Error("Failed to create HIT");
      }

      // Poll for results
      let assignment = null;
      const startTime = Date.now();
      const maxWaitTime = hitValiditySeconds * 1000;
      const pollInterval = 5000; // Poll every 5 seconds

      while (Date.now() - startTime < maxWaitTime) {
        const listAssignmentsResponse = await mturkClient.send(
          new ListAssignmentsForHITCommand({
            HITId: hitId,
            AssignmentStatuses: ["Submitted", "Approved"],
          }),
        );

        if (
          listAssignmentsResponse.Assignments &&
          listAssignmentsResponse.Assignments.length > 0
        ) {
          assignment = listAssignmentsResponse.Assignments[0];
          break;
        }

        // Wait before polling again
        await new Promise((resolve) => setTimeout(resolve, pollInterval));
      }

      // Return results
      if (assignment && assignment.AssignmentId) {
        // Auto-approve the assignment
        try {
          await mturkClient.send(
            new ApproveAssignmentCommand({
              AssignmentId: assignment.AssignmentId,
              RequesterFeedback: "Thank you for your response!",
            }),
          );
        } catch (approveError) {
          console.error("Error approving assignment:", approveError);
          // Continue with the response even if approval fails
        }

        if (assignment.Answer) {
          // Parse XML answer (simplified - in production, use an XML parser)
          const answerText = assignment.Answer.replace(
            /<\?xml.*?\?>/,
            "",
          ).replace(
            /<Answer>.*?<QuestionIdentifier>.*?<\/QuestionIdentifier>.*?<FreeText>(.*?)<\/FreeText>.*?<\/Answer>/s,
            "$1",
          );

          return {
            content: [
              {
                type: "text",
                text: `Human response: ${answerText}`,
              },
            ],
          };
        } else {
          return {
            content: [
              {
                type: "text",
                text: `Assignment received but answer format was invalid. Assignment ID: ${assignment.AssignmentId}, HIT ID: ${hitId}`,
              },
            ],
          };
        }
      } else {
        return {
          content: [
            {
              type: "text",
              text: `No response received within the maximum wait time. Your question is still available for workers on MTurk. HIT ID: ${hitId} - You can check its status later with the checkHITStatus tool.`,
            },
          ],
        };
      }
    } catch (error) {
      console.error("Error in askHuman tool:", error);
      return {
        content: [
          {
            type: "text",
            text: `Error: ${error instanceof Error ? error.message : String(error)}`,
          },
        ],
      };
    }
  },
);
@olalonde olalonde added the enhancement New feature or request label May 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant