User Feedback
Logging user feedback is a crucial part of evaluating an LLM app. Even though user feedback is subjective and biased towards negative, it is a valuable source of information to improve the quality of your app.
Setup user feedback in your app to log the user feedback to phospho, review it in the webapp, improve the automatic evaluations, and make your app better.
Architecture: what's the task_id?
In your app, you should collect user feedback after having logged a task to phospho. Every task logged to phospho is identified by a unique task_id.
For phospho to know what task the user is giving feedback on, you need to keep track of the task_id.
There are two ways to manage the task_id: frontend or backend.
Any way you chose, there are helpers in the phospho package to make it easier.
Option 1: Task id managed by Frontend
- In your frontend, you generate a task id using UUID V4
- You pass this task id to your backend. The backend executes the task and log the task to phospho with this task id.
- In your frontend, you collect user feedback based on this task id.
Option 2: Task id managed by Backend
- In your frontend, you ask your backend to execute a task.
- The backend generates a task id using UUID V4, and logs the task to phospho with this task id.
- The backend returns the task id to the frontend.
- In your frontend, you collect user feedback based on this task id.
Backend: Log to phospho with a known task_id
The phospho package provides multiple helpers to manage the task_id.
Make sure you have initialized the phospho package with your project_id and api_key somewhere in your app.
You can fetch the task_id generated by phospho.log:
logged_content = phospho.log(input="question", output="answer")
task_id: str = logged_content["task_id"]
To generate a new task_id, you can use the new_task function.
task_id: str = phospho.new_task()
#Â Pass it to phospho.log to create a task with this id
phospho.log(input="question", output="answer", task_id=task_id)
To get the latest task_id, you can use the latest_task_id variable.
The phospho package provides multiple helpers to manage the task_id.
Make sure you have initialized the phospho package with your project_id and api_key somewhere in your app.
import { phospho } from "phospho";
phospho.init({ projectId: "your_project_id", apiKey: "your_api_key" });
You can fetch the task_id generated by phospho.log:
const loggedContent = await phospho.log({
input: "question",
output: "answer",
});
const taskId: string = loggedContent.task_id;
To generate a new task_id, you can use the newTask function.
const taskId = phospho.newTask();
// Pass it to phospho.log to create a task with this id
phospho.log({ input: "question", output: "answer", taskId: taskId });
To get the latest task_id, you can use the latestTaskId variable.
When using the API directly, you need to manage the task_id by yourself.
Create a task_id by generating a string hash. It needs to be unique for each task.
Pass this task_id to the log endpoint.
Frontend: Collect user feedback
Once your backend has executed the task and logged it to phospho with a known task_id, send the task_id back to your frontend.
In your frontend, using the task_id, you can collect user feedback and send it to phospho.
We provide React components to kickstart your user feedback collection in your app.
import "./App.css";
import { FeedbackDrawer, Feedback } from "phospho-ui-react";
import "phospho-ui-react/dist/index.css";
function App() {
return (
<div className="App">
<header className="App-header">
<FeedbackDrawer
// Get your project_id on phospho
projectId="..."
// The task_id logged to phospho. Fetch it from your backend after logging
taskId="..."
// Source will be also logged to phospho
source={"user_feedback"}
// Customize the drawer
title="Send Feedback"
description="Help us improve our product."
onSubmit={(feedback: Feedback) =>
console.log("Submitted: ", feedback)
}
onClose={(feedback: Feedback) => console.log("Closed: ", feedback)}
/>
</header>
</div>
);
}
export default App;
In the browser, use the sendUserFeedback function. This function doesn't need your phospho api key. This is done to avoid leaking your phospho API key. However, this function still requires the projectId.
Here is how to use the sendUserFeedback function.
import { sendUserFeedback } from "phospho";
// Handle logging in your backend, and send the task_id to the browser
const taskId = await fetch("https://your-backend.com/some-endpoint", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
your: "stuff",
}),
})
.then((res) => res.json())
.then((data) => data.task_id);
// When you collect feedback, send it to phospho
// For example, when the user clicks on a button
sendUserFeedback({
projectId: "your_project_id",
tastId: taskId,
flag: "success", // or "failure"
source: "user",
notes: "Some notes (can be None)",
});
If you are using a different language or a different way to manage the frontend, you can use the API endpoint tasks/{task-id}/flag directly.
This endpoint is public. You only need to pass the task_id and project_id. This is done to avoid leaking your phospho API key.
Backend: Manage user feedback collection
If you don't want to collect user feedback in the frontend, you can instead create an endpoint in your backend and collect user feedback there.
The phospho python package provides a user_feedback function to log user feedback.
The phospho javascript module provides a userFeedback function to log user feedback.
You can use the API endpoint tasks/{task-id}/flag directly.