• Build the agent harness for planning, tool use, memory, context compaction, interruption and recovery during long-running work.
• Integrate OpenAI and Anthropic model APIs. Select models using measured quality, latency and cost, with clear failure handling.
• Evaluate existing harnesses and build only the missing pieces. Keep the harness connected to AWS workflow execution and approval state.
• Keep authorization and sensitive writes behind application policy checks. Require evidence of the resulting state before marking work complete.
• Build evaluation datasets and replay tests for valid findings, access decisions, app setup and AI agent onboarding.
• Measure false positives, missed risks, unsafe actions, completion rate and cost. Turn production failures into regression tests.
• Test prompt injection, stale evidence, conflicting instructions and unsafe tool use. Protect tenant boundaries in retrieval and memory.
• Improve prompts, retrieval, tools and model routing. Evaluate fine-tuning or other post-training methods when results justify them and data use is approved.