--- schema_version: "1" id: 46bea95214c84b45ccf2997a3c110afb81486165b6eb8261ab12456aa1d0e009 title: Troubleshoot Agent Networks collection: mulesoft version: current locale: en-us portal_url: https://docs.salesforce.com/mulesoft/current/en-us/agent-network/latest/af-troubleshoot-agent-networks.md canonical_url: https://docs.mulesoft.com/agent-network/latest/af-troubleshoot-agent-networks source_site: docs.mulesoft.com content_type: documentation content_hash: 000dd47f795d47c16f1f3cdf4789a44fb5cda389e27c40f7cfebe16f49579339 description: MuleSoft Documentation Site release: "2.0" latest: true --- --- product: Building Agent Networks version: 2.0 is-latest-version: true --- # Troubleshoot Agent Networks > For the full documentation index, see: https://docs.mulesoft.com/llms.txt Resolve common agent network errors and connectivity issues by enabling verbose logging, message logging policies, and distributed tracing. Using logs and distributed traces, you can observe agents and MCP servers and see how requests flow through brokers, which tools and policies apply, and where failures or latency originate. For Anypoint CLI Agent Fabric plugin errors, see [Troubleshoot Agent Fabric CLI Errors](https://docs.mulesoft.com/agent-network/latest/af-troubleshoot-errors). > [!NOTE] Use Agent Visualizer to > [view metrics, logs, and traces](https://docs.mulesoft.com/agent-visualizer/troubleshoot-agent-network) for agents and > MCP servers in your network. ## Enable Logging to Debug Agent Brokers Enable logging and log levels from CloudHub 2.0 or Runtime Fabric. Agent brokers provide two levels of logging detail: - **INFO**: Logs node transitions, MCP tool discovery, agent card retrieval, and related operational events. INFO-level logs don’t use the `INSECURE-LOGGING` level. - **DEBUG**: Logs LLM reasoning, LLM input and output, tools invoked, and all external calls made by the broker. All DEBUG-level logs use the `INSECURE-LOGGING` level. To enable logging: 1. In Runtime Manager, navigate to the application for the agent broker. 2. Choose one of these options to configure logging: - [CloudHub 2.0](https://docs.mulesoft.com/cloudhub-2/ch2-manage-app-logs#configure-log-levels-and-categories) - [Runtime Fabric](https://docs.mulesoft.com/runtime-fabric/latest/deploy-to-runtime-fabric#configure-logging-and-log-levels) 3. To view operational logs, set the log level to `INFO`. 4. To view detailed LLM and tool execution logs, set the log level for `INSECURE-LOGGING` to `DEBUG`. > [!IMPORTANT] `INSECURE-LOGGING` logs content that can contain sensitive information, such as LLM > reasoning, and inputs and outputs from agents and assets defined in your agent network. This > content is very likely to contain personally identifiable information (PII). Disable > `INSECURE-LOGGING` after you finish troubleshooting. ## Enable Message Logging Policy Edit `agent-network.yaml` to set message logging for MCP servers and agents. Message logging uses the [Omni Gateway Message Logging policy](https://docs.mulesoft.com/gateway/latest/policies-included-message-logging). In the `connection` section of the agent or MCP server instance you want to log, add this logging policy inside `policies`: ```yaml policies: ## add the following inside the policy setting - ref: name: message-logging namespace: business_group_UUID configuration: loggingConfiguration: - itemName: "Payload" itemData: message: "#[payload]" firstSection: true secondSection: true level: "INFO" - itemName: "Headers" itemData: message: "#[attributes]" firstSection: true secondSection: true level: "INFO" ``` > [!NOTE] On non-US control planes, the value for the `namespace` parameter can change. In Exchange, > check the business group in the API Gateway Message Logging Policy Template. ## Troubleshooting Common Issues and Errors Use the steps in this section to diagnose and fix common problems in agent networks. ### Issue: Traces Not Shown If Anypoint Monitoring isn’t showing traces for agents and MCP servers, check these settings: - In API Manager, verify you’ve enabled the tracing policy for all brokers and agents. - In the managed Omni Gateways, verify that you’ve enabled Distributed Tracing for ingress and egress. ### Issue: No Agent Network Commands in Anypoint Code Builder If you installed Anypoint Code Builder successfully, but you don’t see agent network commands in the interface, verify that you have the latest version of the Anypoint Extension Pack. Then, restart VS Code. ### Issue: Linting Errors in Agent Script File If you see linting errors in your `.agent` file in VS Code, you likely have the Agentforce DX extension pack installed, and it’s conflicting with Anypoint Code Builder. This extension pack includes a dependency on the Agent Script extension that conflicts with the Agent Script extension built into Anypoint Code Builder. To resolve this conflict, uninstall the Agentforce DX extension pack from VS Code and restart the IDE. Anypoint Code Builder already includes all Agent Script functionality, so you don’t need a separate extension. ### Issue: No Agent Network Deployment Targets If you attempt to deploy an agent network, but you don’t see any deployment targets in Anypoint Code Builder or Anypoint CLI, then you need to set up your target deployment space and set up gateways. For instructions, see [Get Started with Agent Networks](https://docs.mulesoft.com/agent-network/latest/af-get-started). ### Issue: Changes Don’t Take Effect After Redeployment If you update `agent-network.yaml` or any `.agent` files but changes aren’t reflected after you redeploy, the deployment used the last compiled `target/` directory. Every source change requires a full rebuild. After any source file change, run the full pipeline: 1. Build 2. Publish 3. Deploy ### Error: "Asset belongs to another project" (Error 2006) If the publish command fails with this error (code `2006`), an asset in your project has the same name as an asset that belongs to a different project in the same organization. Asset names must be unique across all projects in an org, and you can’t reuse an asset name that another project already owns. To resolve this conflict: - In `agent-network.yaml`, rename the conflicting asset to a unique name. - Check the other project referenced in the error to verify which assets it owns. If you own the conflicting project, remove or rename the asset in that project before you republish. ### Error: CrashLoopBackOff or Exit 139 with No Clear Error If your deployment keeps restarting without a clear error, the broker is most likely crashing on a bad upstream connection before your agent network configuration is parsed. Common causes include: - An expired API key - An incorrect MCP server URL - An MCP server returning JSON instead of SSE - A missing A2A agent card Before you deploy, validate every connection with live probes. Don’t troubleshoot your agent network configuration until you confirm all upstream connections are healthy. ### Issue: Broker Cannot Reach Downstream Agents (Last-Mile Security Enabled) If your broker deploys successfully and answers general questions, but fails whenever it routes a request to a downstream agent or MCP server, check if **Last-Mile Security** is enabled on the Omni Gateway used for outbound (egress) traffic. Agent networks route outbound traffic directly to the pod through the egress gateway (port 8082), which bypasses the ingress-to-pod hop that Last-Mile Security protects. As a result, agent networks are not compatible with Last-Mile Security on the egress gateway. To resolve: - In Runtime Manager, go to **Omni Gateways** > > **Settings**, and disable **Last-Mile Security**. - If Last-Mile Security is required for other applications on that gateway, deploy your agent network to a separate egress gateway that has Last-Mile Security disabled. > [!NOTE] This is a behavior change from Agent Network version 1.0, which routed outbound traffic > through the CloudHub ingress gateway (port 8081) and was therefore unaffected by Last-Mile > Security. A gateway that worked under version 1.0 with Last-Mile Security enabled will fail under > version 2.0. ### Error: "Cannot complete task due to issue accessing reasoning engine" This is a generic error that the broker issues when the LLM fails. To troubleshoot this error, [enable the message logging policy](#enable-message-logging) on the LLM instance, and then review the logs to determine the cause of the error. ### Issue: LLM Responds Without Calling the Agent or Tool If the broker responds with information from the LLM’s training data instead of the LLM delegating to your agent or calling an MCP tool, the LLM’s instructions aren’t explicit enough about delegation. This is especially common with Gemini on widely-known topics. In the broker’s system instructions: - Add a hard directive, such as `MUST delegate IMMEDIATELY` and `NEVER answer yourself`. - Map every intent to a specific named action. ### Error: Runtime System Limit Exceeded If your agent network encounters errors related to exceeding iteration limits, tool call limits, or state size limits, you can configure runtime system limits in your `exchange.json` file. These variables control graph execution behavior and help prevent infinite loops or excessive resource usage. Common limit-related errors include: - Maximum handoff iterations exceeded - Maximum reasoning iterations exceeded - Maximum tool call count exceeded - State size limit exceeded To adjust these limits, add runtime system limit variables to the `metadata.variables` section of your `exchange.json` file. For detailed information about configuring these limits, see [Runtime System Limits Configuration](https://docs.mulesoft.com/agent-network/latest/af-agent-network-yaml-reference#runtime-system-limits). ### Error: "HTTP error 404: Agent card not found at /.well-known/agent-card.json" You can encounter this error in two different scenarios: 1. The agent card URL is incorrect. In this case: - Check that the URL you set on the connection ends with a `/`. - Use a browser or Postman to request the agent card URL by appending `.well-known/agent-card.json` to the connection URL, for example, `[https://ping-id-agent-url/pingid/.well-known/agent-card.json](https://ping-id-agent-url/pingid/.well-known/agent-card.json)`. 2. The A2A agent doesn’t use A2A protocol version 1.0.0. In this case: - If you created the agent with MuleSoft, verify that you are using the latest version of the Anypoint Connector for A2A (A2A Connector). - If you are using an external agent, verify that it uses the A2A protocol version 1.0.0. ### Error: "401 Unauthorized" When a Broker Calls an Agent If the broker returns **401 Unauthorized** error when it routes a request to a downstream A2A agent, the A2A protocol version declared for that agent in `agent-network.yaml` might not match the agent’s actual runtime protocol version. When the declared version doesn’t match the real version, the platform applies the wrong A2A Agent Card policy. That policy doesn’t rewrite the agent card’s `url` field to the egress proxy URL, so the A2A SDK posts `send_message` directly to the external gateway URL and bypasses the egress proxy. Because the egress proxy is what injects the `client_id` and `client_secret` headers required by the Client-ID-Enforcement policy, the request arrives without credentials and is rejected with a `401` error. To resolve this error: 1. Confirm the agent’s real A2A protocol version. Request the agent card in a browser or Postman by appending `.well-known/agent-card.json` to the connection URL, and check the `protocolVersion` field. 2. Update the connection definition in `agent-network.yaml` so the declared protocol version matches the agent’s real version. 3. Rebuild, republish, and redeploy the agent network. > [!NOTE] A protocol version mismatch surfaces as an authentication (401) error, not as a version > error. Verify the protocol version whenever a downstream agent call fails with `401` even if > you’ve correctly configured the the egress auth policy. ## See Also - [Known Issues](https://help.salesforce.com/s/issues)