How to use the AgentCoreCodeInterpreter tool to generate graphs, visualizations?

[cmurtaugh]

I'm struggling to get the AgentCoreCodeInterpreter tool to work with a Strands agent in what I expect would be a fairly common use case. I have an agent that, among other things, I'd like to be able to call the code interpreter in order to produce graphs/visualizations of data and to perform more complex analysis. I can get the agent to call the code interpreter, and it appears to generate images, but the agent cannot successfully do anything with the files that are produced.

I've stripped down my code to be as simple as possible.

Code:

from strands import Agent
from strands_tools.code_interpreter import AgentCoreCodeInterpreter
from strands_tools import file_read, file_write
from strands.agent import AgentResult

# Initialize the Code Interpreter tool
code_interpreter_tool = AgentCoreCodeInterpreter(region="us-east-1")

# Define the agent's system prompt
SYSTEM_PROMPT = """You are an AI assistant that validates answers, performs analysis, and generates visualizations through code execution.
When asked about code, algorithms, or calculations, write Python code to verify your answers.
If you need to perform calculations, generate data, or create visualizations, write and execute Python code using the code_interpreter tool.
If you need to read from or write to files, use the file_read and file_write tools.
If the code_interpreter tool creates files, such as images or data files, read them from the sandbox and save them locally using the file_write tool.
Provide clear explanations alongside any code you generate.
"""

# Create an agent with the Code Interpreter tool
agent = Agent(
    tools=[code_interpreter_tool.code_interpreter, file_read, file_write],
    system_prompt=SYSTEM_PROMPT,
)

# Test the agent with a sample prompt
prompt = "Create me a pie chart with three random segments. Save the chart locally and provide the file path."
print(f"\n\nPrompt: {prompt}\n\n")

response: AgentResult = agent(prompt)

When I run this code the agent struggles to retrieve the image file data from the sandbox, flailing with various attempts to retrieve base64-encoded data, deal with JSON formatting, etc. I almost always see an error that says bedrock threw context window overflow error.

Output:

Prompt: Create me a pie chart with three random segments. Save the chart locally and provide the file path.


I'll create a pie chart with three random segments using Python and save it locally. Let me generate the chart with random data and colors.
Tool #1: code_interpreter
Now let me read the chart from the sandbox and save it locally:
Tool #2: file_read
Let me check what files are available in the current directory:
Tool #3: code_interpreter
Great! The file was created. Now let me read the image file and save it locally:
Tool #4: code_interpreter
bedrock threw context window overflow error
Let me try a different approach - I'll use a shell command to copy the file and then read it in binary mode:
Tool #5: code_interpreter

Tool #6: code_interpreter
Let me fix the formatting:
Tool #7: code_interpreter

Tool #8: code_interpreter
Let me try a simpler approach by using the executeCommand action:
Tool #9: code_interpreter
Let me try a different approach by using Python to save the raw binary data:
Tool #10: code_interpreter
Now I'll use a Python script to create a smaller version of the image that can be more easily transferred:
Tool #11: code_interpreter
I'm having issues with the JSON formatting. Let me use a simpler approach and just create the base64 encoded data:
Tool #12: code_interpreter
Let me try a different approach - I'll create a text file with the binary data information and then use the file_read tool to access it directly:
Tool #13: code_interpreter
Great! Now let me read the base64 data and save the image locally:
Tool #14: file_read

Tool #15: code_interpreter
Let me use the correct format for listing files:
Tool #16: code_interpreter
Let me try using the executeCommand to list files and then directly copy the image:
Tool #17: code_interpreter
I apologize for the formatting issues. Let me try a simpler approach by creating the image content as a hex dump that can be more easily handled:
Tool #18: code_interpreter
Let me try with a properly formatted JSON:
Tool #19: code_interpreter
Now let me decode the base64 and save the image file locally using the file_write tool. First, let me get the base64 data:
Tool #20: code_interpreter
Let me try a simpler approach and just get the base64 data:
Tool #21: code_interpreter
I'm having persistent formatting issues. Let me try a different approach by creating the image fresh and using a more direct method:
Tool #22: code_interpreter
Let me recreate the chart with a simpler approach and proper JSON formatting:
Tool #23: code_interpreter
I see there's a persistent issue with the JSON formatting. Let me try a much simpler version:
Tool #24: code_interpreter
I apologize for the technical difficulties with the JSON formatting. Let me provide you with a summary of what we accomplished and create the file using a different approach.

From our successful execution earlier, I created a pie chart with these specifications:

- **Values**: 61, 24, 81 (representing the three random segments)
- **Labels**: Segment A (36.7%), Segment B (14.5%), Segment C (48.8%)
- **Colors**: Purple (#d402fb), Cyan (#44dfec), Orange (#d39d17)
- **File**: random_pie_chart.png (129,403 bytes)

Since I'm experiencing issues with transferring the binary data directly from the sandbox, let me recreate a simpler version that I can save locally:
Tool #25: file_write
Do you want to proceed with the file write? [y/*] y
Perfect! I've created a Python script that you can run to generate the pie chart locally. Here's what I've provided:

## File Created: `pie_chart_code.py`

**File Path**: `pie_chart_code.py` (saved in your current directory)

This Python script will create a pie chart with three random segments when you run it:

### Chart Details:
- **Segment A**: 61 units (36.7%) - Purple color (#d402fb)  
- **Segment B**: 24 units (14.5%) - Cyan color (#44dfec)
- **Segment C**: 81 units (48.8%) - Orange color (#d39d17)

### To use the file:
1. Make sure you have matplotlib installed: `pip install matplotlib`
2. Run the script: `python pie_chart_code.py`
3. The chart will be saved as `random_pie_chart.png` in the same directory

The chart will be a high-quality PNG image (300 DPI) with a clean, professional appearance showing the three random segments with their respective percentages and colors.

Is there somewhere I can find an example of how to get an agent to call an AgentCoreCodeInterpreter tool properly?