ragas官方文档中文版（六十七）

操作指南本节中的每个指南都针对您作为有经验的用户在使用 Ragas 时可能遇到的实际问题提供了专注的解决方案。这些指南设计得简洁直接为您的问题提供快速解决方案。我们假设您对 Ragas 的概念有基本了解且能够熟练使用。如果不是请先浏览 快速入门 Get Started部分。使用 Llama 4 评估 LlamaStack Web Search 接地性在本教程中我们将测量 LlamaStack 网页搜索智能体生成响应的接地性。LlamaStack 是由 Meta 维护的开源框架用于简化大语言模型驱动应用的开发和部署。评估将使用 Ragas 指标和 Meta Llama 4 Maverick 作为评判模型进行。设置并运行 LlamaStack 服务器此命令安装使用 Together 推理提供程序的 LlamaStack 服务器所需的所有依赖项。使用 conda 命令!pipinstallragas langchain-together uv!uv run--withllama-stack llama stack build--templatetogether --image-type conda使用 venv 命令!pipinstallragas langchain-together uv!uv run--withllama-stack llama stack build--templatetogether --image-type venvimportosimportsubprocessdefrun_llama_stack_server_background():log_fileopen(llama_stack_server.log,w)processsubprocess.Popen(uv run --with llama-stack llama stack run together --image-type venv,shellTrue,stdoutlog_file,stderrlog_file,textTrue,)print(fStarting LlamaStack server with PID:{process.pid})returnprocessdefwait_for_server_to_start():importrequestsfromrequests.exceptionsimportConnectionErrorimporttime urlhttp://0.0.0.0:8321/v1/healthmax_retries30retry_interval1print(Waiting for server to start,end)for_inrange(max_retries):try:responserequests.get(url)ifresponse.status_code200:print(\nServer is ready!)returnTrueexceptConnectionError:print(.,end,flushTrue)time.sleep(retry_interval)print(\nServer failed to start after,max_retries*retry_interval,seconds)returnFalse# use this helper if needed to kill the serverdefkill_llama_stack_server():# Kill any existing llama stack server processesos.system(ps aux | grep -v grep | grep llama_stack.distribution.server.server | awk {print $2} | xargs kill -9)启动 LlamaStack 服务器server_processrun_llama_stack_server_background()assertwait_for_server_to_start()Starting LlamaStack serverwithPID:95508Waitingforserver to start....Serverisready!构建搜索智能体fromllama_stack_clientimportLlamaStackClient,Agent,AgentEventLogger clientLlamaStackClient(base_urlhttp://0.0.0.0:8321,)agentAgent(client,modelmeta-llama/Llama-3.1-8B-Instruct,instructionsYou are a helpful assistant. Use web search tool to answer the questions.,tools[builtin::websearch],)user_prompts[In which major did Demis Hassabis complete his undergraduate degree? Search the web for the answer.,Ilya Sutskever is one of the key figures in AI. From which institution did he earn his PhD in machine learning? Search the web for the answer.,Sam Altman, widely known for his role at OpenAI, was born in which American city? Search the web for the answer.,]session_idagent.create_session(test-session)forpromptinuser_prompts:responseagent.create_turn(messages[{role:user,content:prompt,}],session_idsession_id,)forloginAgentEventLogger().log(response):log.print()现在让我们深入了解智能体的执行步骤看看我们的智能体表现如何。session_responseclient.agents.session.retrieve(session_idsession_id,agent_idagent.agent_id,)评估智能体响应我们希望测量 LlamaStack 网页搜索智能体生成响应的接地性。为此我们需要 EvaluationDataset 和指标来评估接地响应。Ragas 提供了一系列现成的指标可用于测量检索和生成的各个方面。用于测量响应接地性的指标忠实度Faithfulness响应接地性Response Groundedness构建 Ragas 评估数据集要使用 Ragas 进行评估我们将创建一个 EvaluationDataset 。importjson# This function extracts the search results for the trace of each querydefextract_retrieved_contexts(turn_object):results[]forstepinturn_object.steps:ifstep.step_typetool_execution:tool_responsesstep.tool_responsesforresponseintool_responses:contentresponse.contentifcontent:try:parsed_resultjson.loads(content)results.append(parsed_result)exceptjson.JSONDecodeError:print(Warning: Unable to parse tool response content as JSON.)continueretrieved_context[]forresultinresults:top_content_list[item[content]foriteminresult[top_k]]retrieved_context.extend(top_content_list)returnretrieved_contextfromragas.dataset_schemaimportEvaluationDataset samples[]references[Demis Hassabis completed his undergraduate degree in Computer Science.,Ilya Sutskever earned his PhD from the University of Toronto.,Sam Altman was born in Chicago, Illinois.,]fori,turninenumerate(session_response.turns):samples.append({user_input:turn.input_messages[0].content,response:turn.output_message.content,reference:references[i],retrieved_contexts:extract_retrieved_contexts(turn),})ragas_eval_datasetEvaluationDataset.from_list(samples)ragas_eval_dataset.to_pandas()user_inputretrieved_contextsresponsereference0In which major did Demis Hassabis complete his…[Demis Hassabis holds a Bachelor’s degree in C…Demis Hassabis completed his undergraduate deg…Demis Hassabis completed his undergraduate deg…1Ilya Sutskever is one of the key figures in AI…[Jump to content Main menu Search Donate Creat…Ilya Sutskever earned his PhD in machine learn…Ilya Sutskever earned his PhD from the Univers…2Sam Altman, widely known for his role at OpenA…[Sam AltmanBiography, OpenAI, Microsoft, …Sam Altman was born in Chicago, Illinois, USA.设置 Ragas 指标fromragas.metricsimportAnswerAccuracy,Faithfulness,ResponseGroundednessfromlangchain_togetherimportChatTogetherfromragas.llmsimportLangchainLLMWrapper llmChatTogether(modelmeta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8,)evaluator_llmLangchainLLMWrapper(llm)ragas_metrics[AnswerAccuracy(llmevaluator_llm),Faithfulness(llmevaluator_llm),ResponseGroundedness(llmevaluator_llm),]评估最后让我们运行评估。fromragasimportevaluate resultsevaluate(datasetragas_eval_dataset,metricsragas_metrics)results.to_pandas()fromragasimportevaluate resultsevaluate(datasetragas_eval_dataset,metricsragas_metrics)results.to_pandas()user_inputretrieved_contextsresponsereferencenv_accuracyfaithfulnessnv_response_groundedness0In which major did Demis Hassabis complete his…[Demis Hassabis holds a Bachelor’s degree in C…Demis Hassabis completed his undergraduate deg…Demis Hassabis completed his undergraduate deg…1.01.01.001Ilya Sutskever is one of the key figures in AI…[Jump to content Main menu Search Donate Creat…Ilya Sutskever earned his PhD in machine learn…Ilya Sutskever earned his PhD from the Univers…1.00.50.752Sam Altman, widely known for his role at OpenA…[Sam Altman Biography, OpenAI, Microsoft, …Sam Altman was born in Chicago, Illinois, USA.Sam Altman was born in Chicago, Illinois.1.01.01.00kill_llama_stack_server()