feat: implement Hook Chains runtime integration for self-healing agent mesh MVP (#711)
* feat: implement Hook Chains runtime integration for self-healing agent mesh MVP - Add Hook Chains config loader, evaluator, and dispatcher in src/utils/hookChains.ts - Wire PostToolUseFailure hook dispatch in executePostToolUseFailureHooks() - Wire TaskCompleted hook dispatch in executeTaskCompletedHooks() - Integrate fallback-agent launcher with permission preservation (canUseTool threading) - Add safety hardening for config-read errors (try-catch protection) - Update docs with MVP runtime trigger explanation - Add 10 unit tests and 4 integration tests covering config, rules, guards, and actions This completes the self-healing agent mesh MVP by enabling declarative rule-based responses to tool failures and task completions, with fallback agent spawning, team notification, and capacity warming actions. * Update docs/hook-chains.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/utils/hookChains.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: address PR #711 review blockers for Hook Chains - Gate hook-chain dispatch behind feature('HOOK_CHAINS') and default env gate to off - Remove committed local artifact (agent.log) and ignore it in .gitignore - Revert hook dispatcher signature threading changes for canUseTool - Use ToolUseContext metadata hookChainsCanUseTool for fallback launch permissions - Make spawn_fallback_agent fail explicitly when launcher context is unavailable - Add config cache max age and guard map size limits to bound runtime memory - Update docs and tests for default-off gating and explicit fallback failure --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Urvish L.
GitHub
1
.gitignore
vendored
1
.gitignore
vendored
@@ -11,3 +11,4 @@ CLAUDE.md
|
||||
package-lock.json
|
||||
/.claude
|
||||
coverage/
|
||||
agent.log
|
||||
|
||||
333
docs/hook-chains.md
Normal file
333
docs/hook-chains.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# Hook Chains (Self-Healing Agent Mesh MVP)
|
||||
|
||||
Hook Chains provide an event-driven recovery layer for important workflow failures.
|
||||
When a matching hook event occurs, OpenClaude evaluates declarative rules and can dispatch remediation actions such as:
|
||||
|
||||
- `spawn_fallback_agent`
|
||||
- `notify_team`
|
||||
- `warm_remote_capacity`
|
||||
|
||||
## Disabled-By-Default Rollout
|
||||
|
||||
> **Rollout recommendation:** keep Hook Chains disabled until you validate rules in your environment.
|
||||
>
|
||||
> - Set top-level config to `"enabled": false` initially.
|
||||
> - Enable per environment when ready.
|
||||
> - Dispatch is gated by `feature('HOOK_CHAINS')`.
|
||||
> - Env gate defaults to off unless `CLAUDE_CODE_ENABLE_HOOK_CHAINS=1` is set.
|
||||
|
||||
This keeps existing workflows unchanged while you tune guard windows and action behavior.
|
||||
|
||||
## Feature Overview
|
||||
|
||||
Hook Chains are loaded from a deterministic config file and evaluated on dispatched hook events.
|
||||
|
||||
MVP runtime trigger wiring:
|
||||
|
||||
- `PostToolUseFailure` hooks dispatch Hook Chains with outcome `failed`.
|
||||
- `TaskCompleted` hooks dispatch Hook Chains with outcome:
|
||||
- `success` when completion hooks did not block.
|
||||
- `failed` when completion hooks returned blocking errors or prevented continuation.
|
||||
|
||||
Default config path:
|
||||
|
||||
- `.openclaude/hook-chains.json`
|
||||
|
||||
Override path:
|
||||
|
||||
- `CLAUDE_CODE_HOOK_CHAINS_CONFIG_PATH=/abs/or/relative/path/to/hook-chains.json`
|
||||
|
||||
Global gate:
|
||||
|
||||
- `feature('HOOK_CHAINS')` must be enabled in the build
|
||||
- `CLAUDE_CODE_ENABLE_HOOK_CHAINS=0|1` (defaults to disabled when unset)
|
||||
|
||||
## Safety Guarantees
|
||||
|
||||
The runtime is intentionally conservative:
|
||||
|
||||
- **Depth guard:** chain dispatch is blocked when `chainDepth >= maxChainDepth`.
|
||||
- **Rule cooldown:** each rule can only re-fire after cooldown expires.
|
||||
- **Dedup window:** identical event/action combinations are suppressed for a window.
|
||||
- **Abort-safe behavior:** if the current signal is aborted, actions skip safely.
|
||||
- **Policy-aware remote warm:** `warm_remote_capacity` skips when remote sessions are policy denied.
|
||||
- **Bridge inactive no-op:** `warm_remote_capacity` safely skips when no active bridge handle exists.
|
||||
- **Missing team context safety:** `notify_team` skips with structured reason if no team context/team file is available.
|
||||
- **Fallback launcher safety:** `spawn_fallback_agent` fails with a structured reason when launch permissions/context are unavailable.
|
||||
|
||||
## Configuration Schema Reference
|
||||
|
||||
Top-level object:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"enabled": true,
|
||||
"maxChainDepth": 2,
|
||||
"defaultCooldownMs": 30000,
|
||||
"defaultDedupWindowMs": 30000,
|
||||
"rules": []
|
||||
}
|
||||
```
|
||||
|
||||
### Top-Level Fields
|
||||
|
||||
| Field | Type | Required | Notes |
|
||||
|---|---|---:|---|
|
||||
| `version` | `1` | No | Defaults to `1`. |
|
||||
| `enabled` | `boolean` | No | Global feature switch for this config file. |
|
||||
| `maxChainDepth` | `integer` | No | Global depth guard (default `2`, max `10`). |
|
||||
| `defaultCooldownMs` | `integer` | No | Default rule cooldown in ms (default `30000`). |
|
||||
| `defaultDedupWindowMs` | `integer` | No | Default action dedup window in ms (default `30000`). |
|
||||
| `rules` | `HookChainRule[]` | No | Defaults to `[]`. May be omitted or empty; when no rules are present, dispatch is a no-op and returns `enabled: false`. |
|
||||
|
||||
> **Note:** An empty ruleset is valid and can be used to keep Hook Chains configured but effectively disabled until rules are added.
|
||||
### Rule Object (`HookChainRule`)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "task-failure-recovery",
|
||||
"enabled": true,
|
||||
"trigger": {
|
||||
"event": "TaskCompleted",
|
||||
"outcome": "failed"
|
||||
},
|
||||
"condition": {
|
||||
"toolNames": ["Edit"],
|
||||
"taskStatuses": ["failed"],
|
||||
"errorIncludes": ["timeout", "permission denied"],
|
||||
"eventFieldEquals": {
|
||||
"meta.source": "scheduler"
|
||||
}
|
||||
},
|
||||
"cooldownMs": 60000,
|
||||
"dedupWindowMs": 30000,
|
||||
"maxDepth": 2,
|
||||
"actions": []
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Required | Notes |
|
||||
|---|---|---:|---|
|
||||
| `id` | `string` | Yes | Stable identifier used in telemetry/guards. |
|
||||
| `enabled` | `boolean` | No | Per-rule switch. |
|
||||
| `trigger.event` | `HookEvent` | Yes | Event name to match. |
|
||||
| `trigger.outcome` | `"success"|"failed"|"timeout"|"unknown"` | No | Single outcome matcher. |
|
||||
| `trigger.outcomes` | `Outcome[]` | No | Multi-outcome matcher. Use either `outcome` or `outcomes`. |
|
||||
| `condition` | `object` | No | Optional extra matching constraints. |
|
||||
| `cooldownMs` | `integer` | No | Overrides global cooldown for this rule. |
|
||||
| `dedupWindowMs` | `integer` | No | Overrides global dedup for this rule. |
|
||||
| `maxDepth` | `integer` | No | Per-rule depth cap. |
|
||||
| `actions` | `HookChainAction[]` | Yes | One or more actions to execute in order. |
|
||||
|
||||
### Condition Fields
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `toolNames` | `string[]` | Matches `tool_name` / `toolName` in event payload. |
|
||||
| `taskStatuses` | `string[]` | Matches `task_status` / `taskStatus` / `status`. |
|
||||
| `errorIncludes` | `string[]` | Case-insensitive substring match against `error` / `reason` / `message`. |
|
||||
| `eventFieldEquals` | `Record<string, string\|number\|boolean>` | Dot-path equality against payload (example: `"meta.source": "scheduler"`). |
|
||||
|
||||
### Actions
|
||||
|
||||
#### `spawn_fallback_agent`
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "spawn_fallback_agent",
|
||||
"id": "fallback-1",
|
||||
"enabled": true,
|
||||
"dedupWindowMs": 30000,
|
||||
"description": "Fallback recovery for failed task",
|
||||
"promptTemplate": "Recover task ${TASK_SUBJECT}. Event=${EVENT_NAME}, outcome=${OUTCOME}, error=${ERROR}. Payload=${PAYLOAD_JSON}",
|
||||
"agentType": "general-purpose",
|
||||
"model": "sonnet"
|
||||
}
|
||||
```
|
||||
|
||||
#### `notify_team`
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "notify_team",
|
||||
"id": "notify-ops",
|
||||
"enabled": true,
|
||||
"dedupWindowMs": 30000,
|
||||
"teamName": "mesh-team",
|
||||
"recipients": ["*"],
|
||||
"summary": "Hook chain ${RULE_ID} fired",
|
||||
"messageTemplate": "Event=${EVENT_NAME} outcome=${OUTCOME}\nTask=${TASK_ID}\nError=${ERROR}\nPayload=${PAYLOAD_JSON}"
|
||||
}
|
||||
```
|
||||
|
||||
#### `warm_remote_capacity`
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "warm_remote_capacity",
|
||||
"id": "warm-bridge",
|
||||
"enabled": true,
|
||||
"dedupWindowMs": 60000,
|
||||
"createDefaultEnvironmentIfMissing": false
|
||||
}
|
||||
```
|
||||
|
||||
## Complete Example Configs
|
||||
|
||||
### 1) Retry via Fallback Agent
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"enabled": true,
|
||||
"maxChainDepth": 2,
|
||||
"defaultCooldownMs": 30000,
|
||||
"defaultDedupWindowMs": 30000,
|
||||
"rules": [
|
||||
{
|
||||
"id": "retry-task-via-fallback",
|
||||
"trigger": {
|
||||
"event": "TaskCompleted",
|
||||
"outcome": "failed"
|
||||
},
|
||||
"cooldownMs": 60000,
|
||||
"actions": [
|
||||
{
|
||||
"type": "spawn_fallback_agent",
|
||||
"id": "spawn-retry-agent",
|
||||
"description": "Retry failed task with fallback agent",
|
||||
"promptTemplate": "A task failed. Recover it safely.\nTask=${TASK_SUBJECT}\nDescription=${TASK_DESCRIPTION}\nError=${ERROR}\nPayload=${PAYLOAD_JSON}",
|
||||
"agentType": "general-purpose",
|
||||
"model": "sonnet"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 2) Notify Only
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"enabled": true,
|
||||
"maxChainDepth": 2,
|
||||
"defaultCooldownMs": 30000,
|
||||
"defaultDedupWindowMs": 30000,
|
||||
"rules": [
|
||||
{
|
||||
"id": "notify-on-tool-failure",
|
||||
"trigger": {
|
||||
"event": "PostToolUseFailure",
|
||||
"outcome": "failed"
|
||||
},
|
||||
"condition": {
|
||||
"toolNames": ["Edit", "Write", "Bash"]
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"type": "notify_team",
|
||||
"id": "notify-team-failure",
|
||||
"recipients": ["*"],
|
||||
"summary": "Tool failure detected",
|
||||
"messageTemplate": "Tool failure detected.\nEvent=${EVENT_NAME} outcome=${OUTCOME}\nError=${ERROR}\nPayload=${PAYLOAD_JSON}"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 3) Combined Fallback + Notify + Bridge Warm
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"enabled": true,
|
||||
"maxChainDepth": 2,
|
||||
"defaultCooldownMs": 45000,
|
||||
"defaultDedupWindowMs": 30000,
|
||||
"rules": [
|
||||
{
|
||||
"id": "full-recovery-chain",
|
||||
"trigger": {
|
||||
"event": "TaskCompleted",
|
||||
"outcomes": ["failed", "timeout"]
|
||||
},
|
||||
"condition": {
|
||||
"errorIncludes": ["timeout", "capacity", "connection"]
|
||||
},
|
||||
"cooldownMs": 90000,
|
||||
"actions": [
|
||||
{
|
||||
"type": "spawn_fallback_agent",
|
||||
"id": "fallback-agent",
|
||||
"description": "Recover failed task execution",
|
||||
"promptTemplate": "Recover failed task and produce a concise fix summary.\nTask=${TASK_SUBJECT}\nError=${ERROR}\nPayload=${PAYLOAD_JSON}"
|
||||
},
|
||||
{
|
||||
"type": "notify_team",
|
||||
"id": "notify-team",
|
||||
"recipients": ["*"],
|
||||
"summary": "Recovery chain triggered",
|
||||
"messageTemplate": "Recovery chain ${RULE_ID} fired.\nOutcome=${OUTCOME}\nTask=${TASK_SUBJECT}\nError=${ERROR}"
|
||||
},
|
||||
{
|
||||
"type": "warm_remote_capacity",
|
||||
"id": "warm-capacity",
|
||||
"createDefaultEnvironmentIfMissing": false
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Template Variables
|
||||
|
||||
The following placeholders are supported by `promptTemplate`, `summary`, and `messageTemplate`:
|
||||
|
||||
- `${EVENT_NAME}`
|
||||
- `${OUTCOME}`
|
||||
- `${RULE_ID}`
|
||||
- `${TASK_SUBJECT}`
|
||||
- `${TASK_DESCRIPTION}`
|
||||
- `${TASK_ID}`
|
||||
- `${ERROR}`
|
||||
- `${PAYLOAD_JSON}`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Rule never triggers
|
||||
|
||||
- Verify `trigger.event` and `trigger.outcome`/`trigger.outcomes` exactly match dispatched event data.
|
||||
- Check `condition` filters (especially `toolNames` and `eventFieldEquals` dot-path keys).
|
||||
- Confirm the config file is valid JSON and schema-valid.
|
||||
|
||||
### Actions show as skipped
|
||||
|
||||
Common skip reasons:
|
||||
|
||||
- `action disabled`
|
||||
- `rule cooldown active ...`
|
||||
- `dedup window active ...`
|
||||
- `max chain depth reached ...`
|
||||
- `No team context is available ...`
|
||||
- `Team file not found ...`
|
||||
- `Remote sessions are blocked by policy`
|
||||
- `Bridge is not active; warm_remote_capacity is a safe no-op`
|
||||
- `No fallback agent launcher is registered in runtime context`
|
||||
|
||||
### Config changes not reflected
|
||||
|
||||
- Loader uses memoization by file mtime/size.
|
||||
- Ensure your editor writes the file fully and updates mtime.
|
||||
- If needed, force reload from the caller side with `forceReloadConfig: true`.
|
||||
|
||||
### Existing workflows changed unexpectedly
|
||||
|
||||
- Set `"enabled": false` at top-level.
|
||||
- Or globally disable with `CLAUDE_CODE_ENABLE_HOOK_CHAINS=0`.
|
||||
- Re-enable gradually after validating one rule at a time.
|
||||
@@ -249,6 +249,11 @@ export type ToolUseContext = {
|
||||
/** When true, canUseTool must always be called even when hooks auto-approve.
|
||||
* Used by speculation for overlay file path rewriting. */
|
||||
requireCanUseTool?: boolean
|
||||
/**
|
||||
* Optional callback used by hook-chain fallback actions that launch
|
||||
* AgentTool from hook runtime paths.
|
||||
*/
|
||||
hookChainsCanUseTool?: CanUseToolFn
|
||||
messages: Message[]
|
||||
fileReadingLimits?: {
|
||||
maxTokens?: number
|
||||
|
||||
@@ -1241,6 +1241,7 @@ async function checkPermissionsAndCallTool(
|
||||
{
|
||||
...toolUseContext,
|
||||
toolUseId: toolUseID,
|
||||
hookChainsCanUseTool: canUseTool,
|
||||
userModified: permissionDecision.userModified ?? false,
|
||||
},
|
||||
canUseTool,
|
||||
@@ -1729,19 +1730,29 @@ async function checkPermissionsAndCallTool(
|
||||
const hookMessages: MessageUpdateLazy<
|
||||
AttachmentMessage | ProgressMessage<HookProgress>
|
||||
>[] = []
|
||||
for await (const hookResult of runPostToolUseFailureHooks(
|
||||
toolUseContext,
|
||||
tool,
|
||||
toolUseID,
|
||||
messageId,
|
||||
processedInput,
|
||||
content,
|
||||
isInterrupt,
|
||||
requestId,
|
||||
mcpServerType,
|
||||
mcpServerBaseUrl,
|
||||
)) {
|
||||
hookMessages.push(hookResult)
|
||||
const hookChainsContext = toolUseContext as ToolUseContext & {
|
||||
hookChainsCanUseTool?: CanUseToolFn
|
||||
}
|
||||
hookChainsContext.hookChainsCanUseTool = canUseTool
|
||||
try {
|
||||
for await (const hookResult of runPostToolUseFailureHooks(
|
||||
toolUseContext,
|
||||
tool,
|
||||
toolUseID,
|
||||
messageId,
|
||||
processedInput,
|
||||
content,
|
||||
isInterrupt,
|
||||
requestId,
|
||||
mcpServerType,
|
||||
mcpServerBaseUrl,
|
||||
)) {
|
||||
hookMessages.push(hookResult)
|
||||
}
|
||||
} finally {
|
||||
if (hookChainsContext.hookChainsCanUseTool === canUseTool) {
|
||||
delete hookChainsContext.hookChainsCanUseTool
|
||||
}
|
||||
}
|
||||
|
||||
return [
|
||||
|
||||
@@ -284,6 +284,7 @@ export async function* runPostToolUseFailureHooks<Input extends AnyObject>(
|
||||
isInterrupt,
|
||||
permissionMode,
|
||||
toolUseContext.abortController.signal,
|
||||
undefined,
|
||||
)) {
|
||||
try {
|
||||
// Check if we were aborted during hook execution
|
||||
|
||||
350
src/utils/hookChains.integration.test.ts
Normal file
350
src/utils/hookChains.integration.test.ts
Normal file
@@ -0,0 +1,350 @@
|
||||
import { afterEach, beforeEach, describe, expect, mock, test } from 'bun:test'
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
|
||||
type HookChainsModule = typeof import('./hookChains.js')
|
||||
|
||||
type ImportHarnessOptions = {
|
||||
allowRemoteSessions?: boolean
|
||||
teamFile?:
|
||||
| {
|
||||
name: string
|
||||
members: Array<{ name: string }>
|
||||
}
|
||||
| null
|
||||
teamName?: string
|
||||
senderName?: string
|
||||
replBridgeHandle?: unknown
|
||||
}
|
||||
|
||||
const tempDirs: string[] = []
|
||||
const originalHookChainsEnabled = process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS
|
||||
|
||||
async function createConfigFile(config: unknown): Promise<string> {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'openclaude-hook-chains-int-'))
|
||||
tempDirs.push(dir)
|
||||
const filePath = join(dir, 'hook-chains.json')
|
||||
await writeFile(filePath, JSON.stringify(config, null, 2), 'utf-8')
|
||||
return filePath
|
||||
}
|
||||
|
||||
async function importHookChainsHarness(
|
||||
options: ImportHarnessOptions = {},
|
||||
): Promise<{
|
||||
mod: HookChainsModule
|
||||
writeToMailboxSpy: ReturnType<typeof mock>
|
||||
agentToolCallSpy: ReturnType<typeof mock>
|
||||
}> {
|
||||
mock.restore()
|
||||
|
||||
const allowRemoteSessions = options.allowRemoteSessions ?? true
|
||||
const teamName = options.teamName ?? 'mesh-team'
|
||||
const senderName = options.senderName ?? 'mesh-lead'
|
||||
const replBridgeHandle = options.replBridgeHandle ?? null
|
||||
|
||||
const writeToMailboxSpy = mock(async () => {})
|
||||
const agentToolCallSpy = mock(async () => ({
|
||||
data: {
|
||||
status: 'async_launched',
|
||||
agentId: 'agent-fallback-1',
|
||||
},
|
||||
}))
|
||||
|
||||
mock.module('../services/analytics/index.js', () => ({
|
||||
logEvent: () => {},
|
||||
}))
|
||||
|
||||
mock.module('./telemetry/events.js', () => ({
|
||||
logOTelEvent: async () => {},
|
||||
}))
|
||||
|
||||
mock.module('../services/policyLimits/index.js', () => ({
|
||||
isPolicyAllowed: () => allowRemoteSessions,
|
||||
}))
|
||||
|
||||
mock.module('./swarm/teamHelpers.js', () => ({
|
||||
readTeamFileAsync: async () => options.teamFile ?? null,
|
||||
}))
|
||||
|
||||
mock.module('./teammateMailbox.js', () => ({
|
||||
writeToMailbox: writeToMailboxSpy,
|
||||
}))
|
||||
|
||||
mock.module('./teammate.js', () => ({
|
||||
getAgentName: () => senderName,
|
||||
getTeamName: () => teamName,
|
||||
getTeammateColor: () => 'blue',
|
||||
}))
|
||||
|
||||
mock.module('../bridge/replBridgeHandle.js', () => ({
|
||||
getReplBridgeHandle: () => replBridgeHandle,
|
||||
}))
|
||||
|
||||
// Integration mock target requested in the task: fallback action can route
|
||||
// through this mocked tool launcher from runtime callback wiring.
|
||||
mock.module('../tools/AgentTool/AgentTool.js', () => ({
|
||||
AgentTool: {
|
||||
call: agentToolCallSpy,
|
||||
},
|
||||
}))
|
||||
|
||||
const mod = await import(`./hookChains.js?integration=${Date.now()}-${Math.random()}`)
|
||||
return { mod, writeToMailboxSpy, agentToolCallSpy }
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS = '1'
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
mock.restore()
|
||||
|
||||
if (originalHookChainsEnabled === undefined) {
|
||||
delete process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS
|
||||
} else {
|
||||
process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS = originalHookChainsEnabled
|
||||
}
|
||||
|
||||
await Promise.all(
|
||||
tempDirs.splice(0).map(dir => rm(dir, { recursive: true, force: true })),
|
||||
)
|
||||
})
|
||||
|
||||
describe('hookChains integration dispatch', () => {
|
||||
test('end-to-end rule evaluation + action dispatch on TaskCompleted failure', async () => {
|
||||
const { mod } = await importHookChainsHarness({
|
||||
teamName: 'mesh-team',
|
||||
senderName: 'mesh-lead',
|
||||
teamFile: {
|
||||
name: 'mesh-team',
|
||||
members: [{ name: 'mesh-lead' }, { name: 'worker-a' }, { name: 'worker-b' }],
|
||||
},
|
||||
})
|
||||
|
||||
const configPath = await createConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'task-failure-recovery',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [
|
||||
{ type: 'spawn_fallback_agent' },
|
||||
{ type: 'notify_team' },
|
||||
],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const spawnSpy = mock(async () => ({ launched: true, agentId: 'agent-e2e-1' }))
|
||||
const notifySpy = mock(async () => ({ sent: true, recipientCount: 2 }))
|
||||
|
||||
const result = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: {
|
||||
task_id: 'task-001',
|
||||
task_subject: 'Patch flaky build',
|
||||
error: 'CI timeout',
|
||||
},
|
||||
},
|
||||
runtime: {
|
||||
onSpawnFallbackAgent: spawnSpy,
|
||||
onNotifyTeam: notifySpy,
|
||||
},
|
||||
})
|
||||
|
||||
expect(result.enabled).toBe(true)
|
||||
expect(result.matchedRuleIds).toEqual(['task-failure-recovery'])
|
||||
expect(result.actionResults).toHaveLength(2)
|
||||
expect(result.actionResults[0]?.status).toBe('executed')
|
||||
expect(result.actionResults[1]?.status).toBe('executed')
|
||||
expect(spawnSpy).toHaveBeenCalledTimes(1)
|
||||
expect(notifySpy).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
test('fallback spawn injects failure context into generated prompt', async () => {
|
||||
const { mod, agentToolCallSpy } = await importHookChainsHarness()
|
||||
|
||||
const configPath = await createConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'fallback-context',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [
|
||||
{
|
||||
type: 'spawn_fallback_agent',
|
||||
description: 'Fallback for failed task',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const result = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: {
|
||||
task_id: 'task-ctx-1',
|
||||
task_subject: 'Repair migration guard',
|
||||
task_description: 'Fix regression in check ordering',
|
||||
error: 'Task failed after retry budget exhausted',
|
||||
},
|
||||
},
|
||||
runtime: {
|
||||
onSpawnFallbackAgent: async request => {
|
||||
const { AgentTool } = await import('../tools/AgentTool/AgentTool.js')
|
||||
await (AgentTool.call as unknown as (...args: unknown[]) => Promise<unknown>)({
|
||||
prompt: request.prompt,
|
||||
description: request.description,
|
||||
run_in_background: request.runInBackground,
|
||||
subagent_type: request.agentType,
|
||||
model: request.model,
|
||||
})
|
||||
return { launched: true, agentId: 'agent-fallback-ctx' }
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
expect(result.actionResults[0]?.status).toBe('executed')
|
||||
expect(agentToolCallSpy).toHaveBeenCalledTimes(1)
|
||||
|
||||
const callInput = agentToolCallSpy.mock.calls[0]?.[0] as {
|
||||
prompt: string
|
||||
description: string
|
||||
run_in_background: boolean
|
||||
}
|
||||
|
||||
expect(callInput.description).toBe('Fallback for failed task')
|
||||
expect(callInput.run_in_background).toBe(true)
|
||||
expect(callInput.prompt).toContain('Event: TaskCompleted')
|
||||
expect(callInput.prompt).toContain('Outcome: failed')
|
||||
expect(callInput.prompt).toContain('Task subject: Repair migration guard')
|
||||
expect(callInput.prompt).toContain('Failure details: Task failed after retry budget exhausted')
|
||||
})
|
||||
|
||||
test('notify_team dispatches mailbox writes when team exists and skips when absent', async () => {
|
||||
const withTeam = await importHookChainsHarness({
|
||||
teamName: 'mesh-a',
|
||||
senderName: 'lead-a',
|
||||
teamFile: {
|
||||
name: 'mesh-a',
|
||||
members: [{ name: 'lead-a' }, { name: 'worker-1' }, { name: 'worker-2' }],
|
||||
},
|
||||
})
|
||||
|
||||
const configPathWithTeam = await createConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'notify-existing-team',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [{ type: 'notify_team' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const withTeamResult = await withTeam.mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPathWithTeam,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-team-ok', error: 'boom' },
|
||||
},
|
||||
})
|
||||
|
||||
expect(withTeamResult.actionResults[0]?.status).toBe('executed')
|
||||
expect(withTeam.writeToMailboxSpy).toHaveBeenCalledTimes(2)
|
||||
|
||||
const recipients = withTeam.writeToMailboxSpy.mock.calls.map(
|
||||
call => call[0] as string,
|
||||
)
|
||||
expect(recipients.sort()).toEqual(['worker-1', 'worker-2'])
|
||||
|
||||
const withoutTeam = await importHookChainsHarness({
|
||||
teamName: 'mesh-missing',
|
||||
senderName: 'lead-missing',
|
||||
teamFile: null,
|
||||
})
|
||||
|
||||
const configPathWithoutTeam = await createConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'notify-missing-team',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [{ type: 'notify_team' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const withoutTeamResult = await withoutTeam.mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPathWithoutTeam,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-team-missing', error: 'boom' },
|
||||
},
|
||||
})
|
||||
|
||||
expect(withoutTeamResult.actionResults[0]?.status).toBe('skipped')
|
||||
expect(withoutTeamResult.actionResults[0]?.reason).toContain('Team file not found')
|
||||
expect(withoutTeam.writeToMailboxSpy).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
test('warm_remote_capacity is a safe no-op when bridge is inactive', async () => {
|
||||
const { mod } = await importHookChainsHarness({
|
||||
allowRemoteSessions: true,
|
||||
replBridgeHandle: null,
|
||||
})
|
||||
|
||||
const configPath = await createConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'bridge-warmup-noop',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [{ type: 'warm_remote_capacity' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const result = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-warm-1' },
|
||||
},
|
||||
})
|
||||
|
||||
expect(result.actionResults).toHaveLength(1)
|
||||
expect(result.actionResults[0]?.status).toBe('skipped')
|
||||
expect(result.actionResults[0]?.reason).toContain('Bridge is not active')
|
||||
})
|
||||
})
|
||||
476
src/utils/hookChains.test.ts
Normal file
476
src/utils/hookChains.test.ts
Normal file
@@ -0,0 +1,476 @@
|
||||
import { afterEach, beforeEach, describe, expect, mock, test } from 'bun:test'
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
|
||||
type HookChainsModule = typeof import('./hookChains.js')
|
||||
|
||||
const tempDirs: string[] = []
|
||||
const originalHookChainsEnabled = process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS
|
||||
|
||||
async function makeConfigFile(config: unknown): Promise<string> {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'openclaude-hook-chains-'))
|
||||
tempDirs.push(dir)
|
||||
const filePath = join(dir, 'hook-chains.json')
|
||||
await writeFile(filePath, JSON.stringify(config, null, 2), 'utf-8')
|
||||
return filePath
|
||||
}
|
||||
|
||||
async function importHookChainsModule(options?: {
|
||||
allowRemoteSessions?: boolean
|
||||
}): Promise<HookChainsModule> {
|
||||
mock.restore()
|
||||
|
||||
const allowRemoteSessions = options?.allowRemoteSessions ?? true
|
||||
|
||||
mock.module('../services/analytics/index.js', () => ({
|
||||
logEvent: () => {},
|
||||
}))
|
||||
|
||||
mock.module('./telemetry/events.js', () => ({
|
||||
logOTelEvent: async () => {},
|
||||
}))
|
||||
|
||||
mock.module('../services/policyLimits/index.js', () => ({
|
||||
isPolicyAllowed: () => allowRemoteSessions,
|
||||
}))
|
||||
|
||||
return import(`./hookChains.js?test=${Date.now()}-${Math.random()}`)
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS = '1'
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
mock.restore()
|
||||
|
||||
if (originalHookChainsEnabled === undefined) {
|
||||
delete process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS
|
||||
} else {
|
||||
process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS = originalHookChainsEnabled
|
||||
}
|
||||
|
||||
await Promise.all(
|
||||
tempDirs.splice(0).map(dir => rm(dir, { recursive: true, force: true })),
|
||||
)
|
||||
})
|
||||
|
||||
describe('hookChains schema validation', () => {
|
||||
test('returns disabled config when env gate is unset', async () => {
|
||||
delete process.env.CLAUDE_CODE_ENABLE_HOOK_CHAINS
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
rules: [
|
||||
{
|
||||
id: 'env-gated-rule',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [{ type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const loaded = mod.loadHookChainsConfig({ pathOverride: configPath })
|
||||
expect(loaded.exists).toBe(false)
|
||||
expect(loaded.config.enabled).toBe(false)
|
||||
expect(loaded.config.rules).toHaveLength(0)
|
||||
})
|
||||
|
||||
test('loads valid config and memoizes by mtime/size', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 5000,
|
||||
defaultDedupWindowMs: 5000,
|
||||
rules: [
|
||||
{
|
||||
id: 'task-failure-fallback',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [
|
||||
{
|
||||
type: 'spawn_fallback_agent',
|
||||
description: 'Fallback recovery agent',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const first = mod.loadHookChainsConfig({ pathOverride: configPath })
|
||||
expect(first.exists).toBe(true)
|
||||
expect(first.error).toBeUndefined()
|
||||
expect(first.fromCache).toBe(false)
|
||||
expect(first.config.enabled).toBe(true)
|
||||
expect(first.config.rules).toHaveLength(1)
|
||||
expect(first.config.rules[0]?.id).toBe('task-failure-fallback')
|
||||
|
||||
const second = mod.loadHookChainsConfig({ pathOverride: configPath })
|
||||
expect(second.exists).toBe(true)
|
||||
expect(second.error).toBeUndefined()
|
||||
expect(second.fromCache).toBe(true)
|
||||
expect(second.config.rules).toHaveLength(1)
|
||||
})
|
||||
|
||||
test('accepts wrapped { hookChains: ... } config shape', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
hookChains: {
|
||||
version: 1,
|
||||
enabled: true,
|
||||
rules: [
|
||||
{
|
||||
id: 'wrapped-shape',
|
||||
trigger: { event: 'PostToolUseFailure', outcomes: ['failed'] },
|
||||
actions: [{ type: 'notify_team' }],
|
||||
},
|
||||
],
|
||||
},
|
||||
})
|
||||
|
||||
const loaded = mod.loadHookChainsConfig({ pathOverride: configPath })
|
||||
expect(loaded.error).toBeUndefined()
|
||||
expect(loaded.config.enabled).toBe(true)
|
||||
expect(loaded.config.rules[0]?.id).toBe('wrapped-shape')
|
||||
})
|
||||
|
||||
test('returns disabled config for invalid schema', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
rules: [
|
||||
{
|
||||
id: 'invalid-rule',
|
||||
trigger: {
|
||||
event: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
outcomes: ['failed'],
|
||||
},
|
||||
actions: [{ type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const loaded = mod.loadHookChainsConfig({ pathOverride: configPath })
|
||||
expect(loaded.exists).toBe(true)
|
||||
expect(loaded.error).toBeDefined()
|
||||
expect(loaded.config.enabled).toBe(false)
|
||||
expect(loaded.config.rules).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('evaluateHookChainRules', () => {
|
||||
test('matches by event + outcome + condition', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const rules = [
|
||||
{
|
||||
id: 'post-tool-failure-rule',
|
||||
trigger: { event: 'PostToolUseFailure', outcome: 'failed' },
|
||||
condition: {
|
||||
toolNames: ['Edit'],
|
||||
errorIncludes: ['permission'],
|
||||
eventFieldEquals: { 'meta.source': 'scheduler' },
|
||||
},
|
||||
actions: [{ type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
]
|
||||
|
||||
const matches = mod.evaluateHookChainRules(rules as never, {
|
||||
eventName: 'PostToolUseFailure',
|
||||
outcome: 'failed',
|
||||
payload: {
|
||||
tool_name: 'Edit',
|
||||
error: 'Permission denied by policy',
|
||||
meta: { source: 'scheduler' },
|
||||
},
|
||||
})
|
||||
|
||||
expect(matches).toHaveLength(1)
|
||||
expect(matches[0]?.rule.id).toBe('post-tool-failure-rule')
|
||||
})
|
||||
|
||||
test('does not match when event/condition fail', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const rules = [
|
||||
{
|
||||
id: 'rule-no-match',
|
||||
trigger: { event: 'PostToolUseFailure', outcomes: ['failed'] },
|
||||
condition: { toolNames: ['Write'] },
|
||||
actions: [{ type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
]
|
||||
|
||||
const wrongEvent = mod.evaluateHookChainRules(rules as never, {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { tool_name: 'Write' },
|
||||
})
|
||||
expect(wrongEvent).toHaveLength(0)
|
||||
|
||||
const wrongCondition = mod.evaluateHookChainRules(rules as never, {
|
||||
eventName: 'PostToolUseFailure',
|
||||
outcome: 'failed',
|
||||
payload: { tool_name: 'Edit' },
|
||||
})
|
||||
expect(wrongCondition).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('dispatchHookChainsForEvent guard logic', () => {
|
||||
test('dedup skips duplicate event/action within dedup window', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 4,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 60_000,
|
||||
rules: [
|
||||
{
|
||||
id: 'dedup-rule',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
cooldownMs: 0,
|
||||
dedupWindowMs: 60_000,
|
||||
actions: [{ id: 'spawn-1', type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const spawn = mock(async () => ({ launched: true, agentId: 'agent-1' }))
|
||||
|
||||
const first = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-123', error: 'boom' },
|
||||
},
|
||||
runtime: { onSpawnFallbackAgent: spawn },
|
||||
})
|
||||
|
||||
const second = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-123', error: 'boom' },
|
||||
},
|
||||
runtime: { onSpawnFallbackAgent: spawn },
|
||||
})
|
||||
|
||||
expect(first.actionResults[0]?.status).toBe('executed')
|
||||
expect(second.actionResults[0]?.status).toBe('skipped')
|
||||
expect(second.actionResults[0]?.reason).toContain('dedup')
|
||||
expect(spawn).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
test('cooldown skips second dispatch when rule cooldown is active', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 4,
|
||||
defaultCooldownMs: 60_000,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'cooldown-rule',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
cooldownMs: 60_000,
|
||||
dedupWindowMs: 0,
|
||||
actions: [{ type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const spawn = mock(async () => ({ launched: true, agentId: 'agent-2' }))
|
||||
|
||||
const first = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-456' },
|
||||
},
|
||||
runtime: { onSpawnFallbackAgent: spawn },
|
||||
})
|
||||
|
||||
const second = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-789' },
|
||||
},
|
||||
runtime: { onSpawnFallbackAgent: spawn },
|
||||
})
|
||||
|
||||
expect(first.actionResults[0]?.status).toBe('executed')
|
||||
expect(second.actionResults[0]?.status).toBe('skipped')
|
||||
expect(second.actionResults[0]?.reason).toContain('cooldown')
|
||||
expect(spawn).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
test('depth limit blocks dispatch when chain depth reaches max', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 1,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'depth-rule',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [{ type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const spawn = mock(async () => ({ launched: true, agentId: 'agent-3' }))
|
||||
|
||||
const result = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-depth' },
|
||||
},
|
||||
runtime: {
|
||||
chainDepth: 1,
|
||||
onSpawnFallbackAgent: spawn,
|
||||
},
|
||||
})
|
||||
|
||||
expect(result.enabled).toBe(true)
|
||||
expect(result.matchedRuleIds).toHaveLength(0)
|
||||
expect(result.actionResults).toHaveLength(0)
|
||||
expect(spawn).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
describe('action dispatch skip scenarios', () => {
|
||||
test('fails spawn_fallback_agent when launcher callback is missing', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'missing-launcher',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [{ type: 'spawn_fallback_agent' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const result = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-missing-launcher' },
|
||||
},
|
||||
runtime: {},
|
||||
})
|
||||
|
||||
expect(result.actionResults[0]?.status).toBe('failed')
|
||||
expect(result.actionResults[0]?.reason).toContain('launcher')
|
||||
})
|
||||
|
||||
test('skips disabled action and does not execute callback', async () => {
|
||||
const mod = await importHookChainsModule()
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'disabled-action-rule',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [
|
||||
{
|
||||
type: 'spawn_fallback_agent',
|
||||
enabled: false,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const spawn = mock(async () => ({ launched: true, agentId: 'agent-4' }))
|
||||
|
||||
const result = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-disabled' },
|
||||
},
|
||||
runtime: { onSpawnFallbackAgent: spawn },
|
||||
})
|
||||
|
||||
expect(result.actionResults[0]?.status).toBe('skipped')
|
||||
expect(result.actionResults[0]?.reason).toContain('disabled')
|
||||
expect(spawn).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
test('skips warm_remote_capacity when policy denies remote sessions', async () => {
|
||||
const mod = await importHookChainsModule({ allowRemoteSessions: false })
|
||||
|
||||
const configPath = await makeConfigFile({
|
||||
version: 1,
|
||||
enabled: true,
|
||||
maxChainDepth: 3,
|
||||
defaultCooldownMs: 0,
|
||||
defaultDedupWindowMs: 0,
|
||||
rules: [
|
||||
{
|
||||
id: 'policy-denied-remote-warm',
|
||||
trigger: { event: 'TaskCompleted', outcome: 'failed' },
|
||||
actions: [{ type: 'warm_remote_capacity' }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const warm = mock(async () => ({
|
||||
warmed: true,
|
||||
environmentId: 'env-123',
|
||||
}))
|
||||
|
||||
const result = await mod.dispatchHookChainsForEvent({
|
||||
configPathOverride: configPath,
|
||||
event: {
|
||||
eventName: 'TaskCompleted',
|
||||
outcome: 'failed',
|
||||
payload: { task_id: 'task-policy-denied' },
|
||||
},
|
||||
runtime: { onWarmRemoteCapacity: warm },
|
||||
})
|
||||
|
||||
expect(result.actionResults[0]?.status).toBe('skipped')
|
||||
expect(result.actionResults[0]?.reason).toContain('policy')
|
||||
expect(warm).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
1518
src/utils/hookChains.ts
Normal file
1518
src/utils/hookChains.ts
Normal file
File diff suppressed because it is too large
Load Diff
@@ -10,6 +10,7 @@ import { wrapSpawn } from './ShellCommand.js'
|
||||
import { TaskOutput } from './task/TaskOutput.js'
|
||||
import { getCwd } from './cwd.js'
|
||||
import { randomUUID } from 'crypto'
|
||||
import { feature } from 'bun:bundle'
|
||||
import { formatShellPrefixCommand } from './bash/shellPrefix.js'
|
||||
import {
|
||||
getHookEnvFilePath,
|
||||
@@ -134,6 +135,7 @@ import { registerPendingAsyncHook } from './hooks/AsyncHookRegistry.js'
|
||||
import { enqueuePendingNotification } from './messageQueueManager.js'
|
||||
import {
|
||||
extractTextContent,
|
||||
createAssistantMessage,
|
||||
getLastAssistantMessage,
|
||||
wrapInSystemReminder,
|
||||
} from './messages.js'
|
||||
@@ -145,6 +147,7 @@ import {
|
||||
import { createAttachmentMessage } from './attachments.js'
|
||||
import { all } from './generators.js'
|
||||
import { findToolByName, type Tools, type ToolUseContext } from '../Tool.js'
|
||||
import type { CanUseToolFn } from '../hooks/useCanUseTool.js'
|
||||
import { execPromptHook } from './hooks/execPromptHook.js'
|
||||
import type { Message, AssistantMessage } from '../types/message.js'
|
||||
import { execAgentHook } from './hooks/execAgentHook.js'
|
||||
@@ -162,9 +165,147 @@ import type { AppState } from '../state/AppState.js'
|
||||
import { jsonStringify, jsonParse } from './slowOperations.js'
|
||||
import { isEnvTruthy } from './envUtils.js'
|
||||
import { errorMessage, getErrnoCode } from './errors.js'
|
||||
import { getAgentName, getTeamName, getTeammateColor } from './teammate.js'
|
||||
import type {
|
||||
HookChainOutcome,
|
||||
HookChainRuntimeContext,
|
||||
SpawnFallbackAgentRequest,
|
||||
SpawnFallbackAgentResponse,
|
||||
} from './hookChains.js'
|
||||
|
||||
const TOOL_HOOK_EXECUTION_TIMEOUT_MS = 10 * 60 * 1000
|
||||
|
||||
function normalizeFallbackAgentModel(
|
||||
model: string | undefined,
|
||||
): 'sonnet' | 'opus' | 'haiku' | undefined {
|
||||
if (model === 'sonnet' || model === 'opus' || model === 'haiku') {
|
||||
return model
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
async function launchFallbackAgentFromHookChains(
|
||||
request: SpawnFallbackAgentRequest,
|
||||
toolUseContext: ToolUseContext,
|
||||
canUseTool: CanUseToolFn,
|
||||
): Promise<SpawnFallbackAgentResponse> {
|
||||
try {
|
||||
const { AgentTool } = await import('../tools/AgentTool/AgentTool.js')
|
||||
const normalizedModel = normalizeFallbackAgentModel(request.model)
|
||||
const result = await AgentTool.call(
|
||||
{
|
||||
prompt: request.prompt,
|
||||
description: request.description,
|
||||
run_in_background: true,
|
||||
...(request.agentType ? { subagent_type: request.agentType } : {}),
|
||||
...(normalizedModel ? { model: normalizedModel } : {}),
|
||||
},
|
||||
toolUseContext,
|
||||
canUseTool,
|
||||
createAssistantMessage({ content: [] }),
|
||||
)
|
||||
|
||||
const data = result.data as
|
||||
| {
|
||||
status?: string
|
||||
agentId?: string
|
||||
agent_id?: string
|
||||
}
|
||||
| undefined
|
||||
const status = data?.status
|
||||
|
||||
if (
|
||||
status === 'async_launched' ||
|
||||
status === 'completed' ||
|
||||
status === 'remote_launched' ||
|
||||
status === 'teammate_spawned'
|
||||
) {
|
||||
return {
|
||||
launched: true,
|
||||
agentId: data?.agentId ?? data?.agent_id,
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
launched: true,
|
||||
reason:
|
||||
status !== undefined
|
||||
? `Fallback launched with status ${status}`
|
||||
: undefined,
|
||||
}
|
||||
} catch (error) {
|
||||
return {
|
||||
launched: false,
|
||||
reason: `Fallback launch failed: ${errorMessage(error)}`,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function dispatchHookChainFromHookRuntime(args: {
|
||||
eventName: 'PostToolUseFailure' | 'TaskCompleted'
|
||||
outcome: HookChainOutcome
|
||||
payload: Record<string, unknown>
|
||||
signal?: AbortSignal
|
||||
toolUseContext?: ToolUseContext
|
||||
}): Promise<void> {
|
||||
try {
|
||||
if (!feature('HOOK_CHAINS')) {
|
||||
return
|
||||
}
|
||||
|
||||
const { dispatchHookChainsForEvent } = await import('./hookChains.js')
|
||||
|
||||
const runtime: HookChainRuntimeContext = {
|
||||
signal: args.signal,
|
||||
senderName: getAgentName() ?? undefined,
|
||||
senderColor: getTeammateColor() ?? undefined,
|
||||
teamName: getTeamName() ?? undefined,
|
||||
}
|
||||
|
||||
const chainDepth = args.toolUseContext?.queryTracking?.depth
|
||||
if (typeof chainDepth === 'number' && Number.isFinite(chainDepth)) {
|
||||
runtime.chainDepth = chainDepth
|
||||
}
|
||||
|
||||
const hookChainsCanUseTool = (
|
||||
args.toolUseContext as
|
||||
| (ToolUseContext & { hookChainsCanUseTool?: CanUseToolFn })
|
||||
| undefined
|
||||
)?.hookChainsCanUseTool
|
||||
|
||||
if (args.toolUseContext) {
|
||||
runtime.onSpawnFallbackAgent = request => {
|
||||
if (!hookChainsCanUseTool) {
|
||||
return Promise.resolve({
|
||||
launched: false,
|
||||
reason:
|
||||
'Fallback action requires canUseTool in this hook runtime context',
|
||||
})
|
||||
}
|
||||
|
||||
return launchFallbackAgentFromHookChains(
|
||||
request,
|
||||
args.toolUseContext!,
|
||||
hookChainsCanUseTool,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
await dispatchHookChainsForEvent({
|
||||
event: {
|
||||
eventName: args.eventName,
|
||||
outcome: args.outcome,
|
||||
payload: args.payload,
|
||||
},
|
||||
runtime,
|
||||
})
|
||||
} catch (error) {
|
||||
logForDebugging(
|
||||
`[hook-chains] Dispatch failed for ${args.eventName}: ${errorMessage(error)}`,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* SessionEnd hooks run during shutdown/clear and need a much tighter bound
|
||||
* than TOOL_HOOK_EXECUTION_TIMEOUT_MS. This value is used by callers as both
|
||||
@@ -3502,9 +3643,11 @@ export async function* executePostToolUseFailureHooks<ToolInput>(
|
||||
): AsyncGenerator<AggregatedHookResult> {
|
||||
const appState = toolUseContext.getAppState()
|
||||
const sessionId = toolUseContext.agentId ?? getSessionId()
|
||||
if (!hasHookForEvent('PostToolUseFailure', appState, sessionId)) {
|
||||
return
|
||||
}
|
||||
const hasPostToolFailureHooks = hasHookForEvent(
|
||||
'PostToolUseFailure',
|
||||
appState,
|
||||
sessionId,
|
||||
)
|
||||
|
||||
const hookInput: PostToolUseFailureHookInput = {
|
||||
...createBaseHookInput(permissionMode, undefined, toolUseContext),
|
||||
@@ -3516,12 +3659,33 @@ export async function* executePostToolUseFailureHooks<ToolInput>(
|
||||
is_interrupt: isInterrupt,
|
||||
}
|
||||
|
||||
yield* executeHooks({
|
||||
hookInput,
|
||||
toolUseID,
|
||||
matchQuery: toolName,
|
||||
let blockingHookCount = 0
|
||||
|
||||
if (hasPostToolFailureHooks) {
|
||||
for await (const result of executeHooks({
|
||||
hookInput,
|
||||
toolUseID,
|
||||
matchQuery: toolName,
|
||||
signal,
|
||||
timeoutMs,
|
||||
toolUseContext,
|
||||
})) {
|
||||
if (result.blockingError) {
|
||||
blockingHookCount++
|
||||
}
|
||||
yield result
|
||||
}
|
||||
}
|
||||
|
||||
await dispatchHookChainFromHookRuntime({
|
||||
eventName: 'PostToolUseFailure',
|
||||
outcome: 'failed',
|
||||
payload: {
|
||||
...hookInput,
|
||||
hook_blocking_error_count: blockingHookCount,
|
||||
hook_execution_skipped: !hasPostToolFailureHooks,
|
||||
},
|
||||
signal,
|
||||
timeoutMs,
|
||||
toolUseContext,
|
||||
})
|
||||
}
|
||||
@@ -3807,12 +3971,36 @@ export async function* executeTaskCompletedHooks(
|
||||
team_name: teamName,
|
||||
}
|
||||
|
||||
yield* executeHooks({
|
||||
let blockingHookCount = 0
|
||||
let preventedContinuation = false
|
||||
|
||||
for await (const result of executeHooks({
|
||||
hookInput,
|
||||
toolUseID: randomUUID(),
|
||||
signal,
|
||||
timeoutMs,
|
||||
toolUseContext,
|
||||
})) {
|
||||
if (result.blockingError) {
|
||||
blockingHookCount++
|
||||
}
|
||||
if (result.preventContinuation) {
|
||||
preventedContinuation = true
|
||||
}
|
||||
yield result
|
||||
}
|
||||
|
||||
await dispatchHookChainFromHookRuntime({
|
||||
eventName: 'TaskCompleted',
|
||||
outcome:
|
||||
blockingHookCount > 0 || preventedContinuation ? 'failed' : 'success',
|
||||
payload: {
|
||||
...hookInput,
|
||||
hook_blocking_error_count: blockingHookCount,
|
||||
hook_prevented_continuation: preventedContinuation,
|
||||
},
|
||||
signal,
|
||||
toolUseContext,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user