# SES Sandboxing Specification for Pattern Execution
## Status: Draft
## Authors
- AI-assisted specification
## Last Updated
2026-01-27
---
## 1. Executive Summary
This specification describes a security architecture for sandboxing untrusted JavaScript execution in the Common Tools pattern runtime using **SES (Secure ECMAScript)** Compartments. The goal is to prevent malicious or buggy pattern code from escaping its sandbox while maintaining high performance by reusing Compartments rather than creating new ones for each invocation.
### Key Principles
1. **Compartment Reuse**: Load compiled pattern modules into a single Compartment per pattern; freeze all exports
2. **No Surviving Closures**: Pattern code must not create closures that persist user data beyond a single invocation
3. **Allowlisted Module-Scope Calls**: Only `pattern`, `pattern`, `lift`, `handler`, and top-level function definitions are permitted at module scope
4. **Frozen Implementations**: All exported `lift` and `handler` implementations are frozen and callable directly
5. **Dynamic Import Isolation**: External ESM imports (esm.sh) get fresh module instances per invocation
---
## 2. Background
### 2.1 Current Architecture
The current execution pipeline:
```
Pattern Source (.tsx)
↓ ts-transformers (compile-time)
↓ js-compiler (TypeScript → AMD bundle)
↓ UnsafeEvalIsolate (direct eval())
↓ instantiateJavaScriptNode() → fn(argument)
```
**Security Gap**: Pattern code currently runs with full access to the JavaScript environment via `eval()`. There are no restrictions on:
- Global access
- Closure creation
- Module imports
- Side effects
### 2.2 Why SES?
SES (Secure ECMAScript) provides:
- **Frozen Intrinsics**: Built-in objects (Array, Object, etc.) are frozen
- **Compartments**: Isolated module graphs with controlled globals
- **Hardened APIs**: `harden()` to deeply freeze object graphs
- **Import Hooks**: Control over module resolution and loading
Alternative considered: QuickJS (via `js-sandbox` package). SES is preferred because:
- Runs in the same V8/SpiderMonkey engine (no serialization overhead)
- Same JavaScript semantics (no edge cases)
- Can share frozen objects between Compartments without copying
- Better debugging experience (same DevTools)
---
## 3. Architecture Overview
### 3.1 High-Level Flow
```
Pattern Source (.tsx)
↓
[1] ts-transformers (enhanced)
- Hoist lift/handler to module scope
- Rewrite inline derive → lift call
- Add __exportName annotations
- Validate allowlisted module-scope calls
↓
[2] js-compiler (existing)
- TypeScript → AMD bundle
↓
[3] SES Compartment Loader (new)
- Create Compartment with frozen globals
- Execute AMD bundle once
- Freeze all exports
- Return callable implementations
↓
[4] Runner (modified)
- Call frozen .implementation directly
- No eval() per invocation
```
### 3.2 Compartment Lifecycle
```
┌─────────────────────────────────────────────────────────────┐
│ Root Compartment (lockdown applied) │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Pattern Compartment (per-pattern, created once) │ │
│ │ │ │
│ │ Globals: { pattern, pattern, lift, handler, ... } │ │
│ │ │ │
│ │ Module Exports (frozen): │ │
│ │ - MyPattern: { implementation: fn, schema: {...} } │ │
│ │ - myLift: { implementation: fn, ... } │ │
│ │ - myHandler: { implementation: fn, ... } │ │
│ │ │ │
│ └──────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ String Eval Compartment (per-string, fresh each time) │ │
│ │ │ │
│ │ Used for: inline strings that couldn't be hoisted │ │
│ │ Created fresh each invocation to prevent closures │ │
│ │ │ │
│ └──────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Dynamic Import Compartment (per-import, fresh) │ │
│ │ │ │
│ │ Used for: await import("https://esm.sh/lodash") │ │
│ │ Fresh instance each time to prevent state leakage │ │
│ │ │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
---
## 4. Transformer Enhancements
### 4.1 Overview of Changes
The `ts-transformers` package requires the following enhancements:
| Transformation | Current Behavior | New Behavior |
|----------------|------------------|--------------|
| `computed(() => ...)` | → `derive({}, () => ...)` | → call to module-scope `lift` |
| `action(() => ...)` | → `handler((_, {}) => ...)({})` | → call to module-scope `handler` |
| inline `derive(input, fn)` | kept inline | → call to module-scope `lift` (or inline if self-contained) |
| `lift(...)` | allowed inline | **ERROR** if not at module scope |
| `handler(...)` | allowed inline | **ERROR** if not at module scope |
| module-scope calls | minimal validation | strict allowlist enforcement |
### 4.2 Module-Scope Allowlist
Only these calls are permitted at module scope:
```typescript
// ALLOWED at module scope
import { pattern, pattern, lift, handler } from "@commontools/common-builder";
// Pattern/Pattern definitions - these call their inner functions at load time
// but user data is not available yet, so this is safe
export const MyPattern = pattern((props) => {
// ...inner function runs at load time...
});
export const MyPattern = pattern("name", (props) => {
// ...inner function runs at load time...
});
// Lift definitions - pure functions, will be frozen
export const myLift = lift((input) => {
return transform(input);
});
// Handler definitions - event handlers, will be frozen
export const myHandler = handler((event, state) => {
return newState;
});
// Top-level function definitions (allowed, but NOT immediately called)
function helperFunction(x: number): number {
return x * 2;
}
// Variable declarations with literals
const CONFIG = { maxItems: 100 };
// Type definitions
type MyType = { name: string };
```
**DISALLOWED** at module scope:
```typescript
// ❌ Immediately calling a function at module scope (creates closures)
const result = someFunction();
// ❌ IIFE (Immediately Invoked Function Expression)
const value = (() => computeSomething())();
// ❌ Any function call except allowlisted builders
const data = fetchData();
// ❌ Await expressions (implies side effects)
const response = await fetch(url);
```
### 4.3 Hoisting Inline Transformations
#### 4.3.1 Computed → Lifted Call
**Before (current):**
```typescript
export const MyPattern = pattern((props) => {
const doubled = computed(() => props.value * 2);
return { doubled };
});
```
**After transformation (current):**
```typescript
export const MyPattern = pattern((props) => {
const doubled = derive({}, () => props.value * 2);
return { doubled };
});
```
**After transformation (new):**
```typescript
// Hoisted to module scope
const __computed_1 = lift<{ value: number }, number>(
({ value }) => value * 2
);
__computed_1.__exportName = "__computed_1"; // Annotation for verification
export const MyPattern = pattern((props) => {
const doubled = __computed_1({ value: props.value });
return { doubled };
});
```
#### 4.3.2 Action → Handler Call
**Before (current):**
```typescript
export const MyPattern = pattern((props) => {
const doSomething = action(() => {
props.count = props.count + 1;
});
return { doSomething };
});
```
**After transformation (new):**
```typescript
// Hoisted to module scope
const __action_1 = handler }>(
(_, { count }) => {
count.set(count.get() + 1);
}
);
__action_1.__exportName = "__action_1";
export const MyPattern = pattern((props) => {
const doSomething = __action_1({ count: props.count });
return { doSomething };
});
```
#### 4.3.3 Inline Derive → Lifted Call
**Before:**
```typescript
export const MyPattern = pattern((props) => {
const total = derive(props.items, items => items.reduce((a, b) => a + b, 0));
return { total };
});
```
**After transformation:**
```typescript
const __derive_1 = lift(
items => items.reduce((a, b) => a + b, 0)
);
__derive_1.__exportName = "__derive_1";
export const MyPattern = pattern((props) => {
const total = __derive_1(props.items);
return { total };
});
```
#### 4.3.4 Optimization: Self-Contained Inline
If the inline function body is entirely self-contained (no references to outer scope), it MAY remain inline for simplicity. The Compartment can evaluate it safely.
**Detection criteria:**
- No free variables (all identifiers are parameters or locally defined)
- No `this` references
- No `arguments` references
- No `eval` or `Function` calls
```typescript
// This can stay inline (self-contained)
const doubled = derive(props.value, x => x * 2);
// This MUST be hoisted (references outer scope)
const doubled = derive(props.value, x => x * multiplier);
```
### 4.4 Export Name Annotation
Every module-scope `pattern`, `pattern`, `lift`, and `handler` must be annotated with its export name. This allows runtime verification that the implementation was indeed defined at module scope and thus frozen.
```typescript
// Before annotation
export const MyLift = lift(fn);
// After annotation (transformer adds this)
export const MyLift = lift(fn);
MyLift.__exportName = "MyLift";
```
For generated/hoisted definitions:
```typescript
const __computed_1 = lift(fn);
__computed_1.__exportName = "__computed_1";
```
**Runtime verification:**
```typescript
function verifyFrozen(impl: any, name: string): void {
if (impl.__exportName !== name) {
throw new Error(`Implementation ${name} was not defined at module scope`);
}
if (!Object.isFrozen(impl.implementation)) {
throw new Error(`Implementation ${name}.implementation is not frozen`);
}
}
```
---
## 5. SES Compartment Integration
### 5.1 Lockdown Configuration
At application startup, apply SES lockdown:
```typescript
import 'ses';
lockdown({
// Error taming: show full stack traces
errorTaming: 'unsafe',
// Stack traces: show real file names
stackFiltering: 'verbose',
// Overrides: allow some taming for compatibility
overrideTaming: 'moderate',
// Console: allow console.log for debugging (configurable)
consoleTaming: 'unsafe', // or 'safe' in production
// Locale: standard behavior
localeTaming: 'unsafe',
// Eval: controlled via Compartments
evalTaming: 'safeEval',
});
```
### 5.2 Pattern Compartment Creation
```typescript
interface PatternCompartment {
compartment: Compartment;
exports: Map;
}
interface FrozenExport {
__exportName: string;
implementation: Function; // frozen
inputSchema: JSONSchema;
resultSchema: JSONSchema;
}
function createPatternCompartment(
compiledAMD: string,
runtimeGlobals: Record
): PatternCompartment {
// Create Compartment with controlled globals
const compartment = new Compartment({
// Frozen intrinsics (automatic with SES)
// Runtime-provided globals (frozen)
...harden(runtimeGlobals),
// Builder functions
pattern: harden(createPatternBuilder()),
pattern: harden(createPatternBuilder()),
lift: harden(createLiftBuilder()),
handler: harden(createHandlerBuilder()),
derive: harden(createDeriveBuilder()),
// Cell/reactive primitives
Cell: harden(Cell),
cell: harden(cell),
// UI helpers (frozen)
h: harden(h),
// Allowlisted globals
console: harden(console), // or filtered console
JSON: harden(JSON),
Math: harden(Math),
});
// Execute the AMD bundle in the Compartment
const moduleExports = compartment.evaluate(compiledAMD)({
// Runtime dependencies injection
"@commontools/common-builder": harden(builderExports),
"@commontools/common-html": harden(htmlExports),
// ... other runtime modules
});
// Freeze all exports deeply
const frozenExports = new Map();
for (const [name, exp] of Object.entries(moduleExports)) {
if (isBuilderExport(exp)) {
// Verify it was defined at module scope
if (exp.__exportName !== name) {
throw new Error(
`Export ${name} was not defined at module scope ` +
`(found __exportName: ${exp.__exportName})`
);
}
// Deep freeze the implementation
harden(exp);
frozenExports.set(name, exp as FrozenExport);
}
}
return { compartment, exports: frozenExports };
}
```
### 5.3 Invoking Frozen Implementations
Once a pattern is loaded and frozen, invocations simply call the frozen functions:
```typescript
class SandboxedRunner {
private patternCompartments = new Map();
async loadPattern(patternId: string, source: string): Promise {
// Compile (existing pipeline)
const compiled = await this.compiler.compile(source);
// Create sandboxed compartment
const patternCompartment = createPatternCompartment(
compiled.js,
this.getRuntimeGlobals()
);
this.patternCompartments.set(patternId, patternCompartment);
}
invoke(patternId: string, exportName: string, input: unknown): unknown {
const pattern = this.patternCompartments.get(patternId);
if (!pattern) {
throw new Error(`Pattern ${patternId} not loaded`);
}
const exp = pattern.exports.get(exportName);
if (!exp) {
throw new Error(`Export ${exportName} not found in ${patternId}`);
}
// Direct call to frozen implementation - no new Compartment needed!
return exp.implementation(input);
}
}
```
### 5.4 String Evaluation Compartment
For strings that couldn't be hoisted (rare case), create a fresh Compartment each time:
```typescript
function evaluateStringInCompartment(
code: string,
globals: Record
): Function {
// Wrap in a function to prevent closure creation
const wrappedCode = `(function(__input__) { return (${code})(__input__); })`;
// Fresh Compartment each time
const compartment = new Compartment({
...harden(globals),
});
// Evaluate and return the wrapper function
return harden(compartment.evaluate(wrappedCode));
}
// Usage
const fn = evaluateStringInCompartment(
'(x) => x * 2',
{ Math: harden(Math) }
);
const result = fn(21); // 42
```
---
## 6. Dynamic Import Support (esm.sh)
### 6.1 Requirements
Patterns may use dynamic imports from esm.sh:
```typescript
export const MyPattern = pattern(async (props) => {
const lodash = await import("https://esm.sh/lodash@4.17.21");
return { result: lodash.capitalize(props.text) };
});
```
**Security requirements:**
1. Each import gets a fresh module instance (no state leakage between invocations)
2. Downloaded code is cached (network efficiency)
3. Module graph is isolated per invocation
### 6.2 Import Hooks
SES Compartments support import hooks for dynamic imports:
```typescript
interface ImportHooks {
resolveHook: (specifier: string, referrer: string) => string;
importHook: (moduleSpecifier: string) => Promise;
}
const esmCache = new Map(); // URL → source code
async function createDynamicImportCompartment(): Promise {
let invocationId = 0;
const compartment = new Compartment({
// ... globals ...
}, {}, {
resolveHook(specifier: string, referrer: string): string {
// Return a unique specifier each time to force fresh instantiation
if (specifier.startsWith('https://esm.sh/')) {
invocationId++;
return `${specifier}#__invocation_${invocationId}`;
}
// Standard resolution for internal modules
return new URL(specifier, referrer).href;
},
async importHook(moduleSpecifier: string): Promise {
// Strip invocation suffix for caching
const url = moduleSpecifier.split('#')[0];
// Check cache
let source = esmCache.get(url);
if (!source) {
// Fetch and cache
const response = await fetch(url);
source = await response.text();
esmCache.set(url, source);
}
// Return as StaticModuleRecord (SES will create fresh instance)
return new StaticModuleRecord(source, moduleSpecifier);
},
});
return compartment;
}
```
### 6.3 Pre-fetching Optimization (Future)
The transformer can analyze dynamic imports and emit prefetch hints:
```typescript
// Transformer output (metadata)
{
dynamicImports: [
"https://esm.sh/lodash@4.17.21",
"https://esm.sh/date-fns@2.30.0"
]
}
// Runner pre-fetch before first invocation
async function prefetchDynamicImports(imports: string[]): Promise {
await Promise.all(imports.map(async (url) => {
if (!esmCache.has(url)) {
const response = await fetch(url);
esmCache.set(url, await response.text());
}
}));
}
```
---
## 7. Closure Prevention Strategy
### 7.1 The Closure Problem
Closures can capture references to user data, leaking it between invocations:
```typescript
// DANGEROUS: Closure captures `userData`
let userData: any;
export const BadPattern = pattern((props) => {
userData = props.secretData; // Captured!
return {
leak: () => userData // Later invocation can access previous user's data!
};
});
```
### 7.2 Prevention Mechanisms
#### 7.2.1 No Module-Scope Mutations
The transformer enforces that module-scope variables:
- Are only assigned at declaration time
- Are never reassigned
- Are const (not let/var)
```typescript
// ❌ REJECTED: let at module scope
let counter = 0;
// ❌ REJECTED: Assignment to module-scope variable
const config = {};
config.key = "value"; // Rejected (mutation)
// ✅ ALLOWED: const with literal/frozen value
const CONFIG = Object.freeze({ key: "value" });
```
#### 7.2.2 Pattern/Pattern Inner Functions Run at Load Time
When `pattern()` or `pattern()` is called, the inner function executes immediately:
```typescript
export const MyPattern = pattern((props) => {
// This code runs at LOAD TIME, not invocation time
// At load time, `props` is a schema placeholder, not user data
return { ui: {props.name}
};
});
```
At load time:
- `props` is a reactive schema placeholder
- No actual user data is available
- The return value defines the reactive graph
At invocation time:
- The reactive graph is already frozen
- User data flows through the frozen graph
- No new closures are created
#### 7.2.3 Frozen Implementations
All `lift` and `handler` implementations are frozen after load:
```typescript
const myLift = lift((input) => input * 2);
// After load: Object.isFrozen(myLift.implementation) === true
// Any attempt to replace the implementation throws
myLift.implementation = evilFn; // TypeError: Cannot assign to read only property
```
#### 7.2.4 Fresh Compartments for Strings
For any code evaluated at runtime (string implementations), a fresh Compartment ensures no closure state persists:
```typescript
// Each invocation gets a fresh Compartment
invocation1: Compartment1 evaluates code → result1
invocation2: Compartment2 evaluates code → result2
// Compartment1 is garbage collected, no state shared
```
---
## 8. Error Handling and Source Map Integration
Debugging sandboxed code presents unique challenges. This section details how to maintain a good developer experience while running code in SES Compartments.
### 8.1 SES Error Taming Options
SES provides configurable "taming" for error objects that controls the security/debuggability trade-off:
#### Safe Mode (`errorTaming: 'safe'`)
```javascript
// Stack traces are sanitized
Error: Something went wrong
at
at
at
```
- File paths, line numbers, and column numbers are hidden
- Prevents attackers from probing system structure via errors
- Error messages may be genericized
- **Use case**: Production with untrusted third-party patterns
#### Unsafe Mode (`errorTaming: 'unsafe'`)
```javascript
// Full stack traces preserved
TypeError: Cannot read property 'map' of undefined
at myLift (/patterns/MyPattern.tsx:42:15)
at invokePattern (runner.ts:1254:12)
at SandboxedRunner.invoke (compartment-manager.ts:89:5)
```
- Real file names and line numbers
- Original error messages intact
- Better debugging experience
- **Use case**: Development, or production with trusted patterns
### 8.2 The Source Map Challenge
Even with `errorTaming: 'unsafe'`, stack traces point to **compiled/transformed code**, not original source:
```
Original TypeScript (MyPattern.tsx)
↓
[ts-transformers] ← Hoisting changes line numbers
↓
[js-compiler] ← TypeScript → JavaScript
↓
[AMD bundling] ← Wraps in AMD loader
↓
Executed in Compartment
```
#### Example: Line Number Mismatch
**Original source (MyPattern.tsx:23):**
```typescript
export const MyPattern = pattern((props) => {
const doubled = computed(() => props.value.map(x => x * 2)); // Line 23
return { doubled };
});
```
**After transformation (compiled.js:5, 47):**
```javascript
// Hoisted to line 5
const __computed_1 = lift(({ value }) => value.map(x => x * 2));
// Original location now at line 47
export const MyPattern = pattern((props) => {
const doubled = __computed_1({ value: props.value });
return { doubled };
});
```
**Error without source mapping:**
```
TypeError: Cannot read property 'map' of undefined
at __computed_1 (eval:5:45) // Points to compiled code!
```
**Error with source mapping:**
```
TypeError: Cannot read property 'map' of undefined
at computed callback (MyPattern.tsx:23:42) // Points to original!
└─ (hoisted to __computed_1)
```
### 8.3 Source Map Preservation Strategy
#### 8.3.1 Compilation Pipeline Source Maps
Each stage produces and consumes source maps:
```
┌─────────────────────────────────────────────────────────────┐
│ Original Source │
│ MyPattern.tsx │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ ts-transformers │
│ Input: MyPattern.tsx │
│ Output: MyPattern.transformed.tsx + sourceMap1 │
│ │
│ sourceMap1: transformed line 5 → original line 23 │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ js-compiler (TypeScript) │
│ Input: MyPattern.transformed.tsx + sourceMap1 │
│ Output: MyPattern.js + sourceMap2 │
│ │
│ sourceMap2: JS line N → transformed line M │
│ (TypeScript compiler can chain source maps) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ AMD Bundler │
│ Input: MyPattern.js + sourceMap2 │
│ Output: bundle.js + sourceMap3 (merged) │
│ │
│ sourceMap3: bundle line P → original line Q │
│ (Merged/chained through all stages) │
└─────────────────────────────────────────────────────────────┘
```
#### 8.3.2 Transformer Source Map Generation
The hoisting transformer must generate accurate source maps:
```typescript
// packages/ts-transformers/src/hoisting.ts
class HoistingTransformer {
private sourceMapGenerator: SourceMapGenerator;
visitComputedCall(node: ts.CallExpression): ts.Expression {
const originalPos = this.sourceFile.getLineAndCharacterOfPosition(
node.getStart()
);
const hoistedName = `__computed_${this.counter++}`;
// Record mapping: hoisted location → original location
this.sourceMapGenerator.addMapping({
generated: { line: this.hoistedLineNumber, column: 0 },
original: { line: originalPos.line + 1, column: originalPos.character },
source: this.sourceFile.fileName,
name: hoistedName,
});
// ... create hoisted node ...
}
}
```
#### 8.3.3 js-compiler Source Map Chaining
The existing js-compiler already supports source maps. Ensure chaining:
```typescript
// packages/js-compiler/typescript/compiler.ts
const compilerOptions: ts.CompilerOptions = {
// ... existing options ...
sourceMap: true,
inlineSources: true, // Include original source in map
inlineSourceMap: false, // Keep separate for chaining
};
// When transformer provides input source map, chain them
if (inputSourceMap) {
// Use source-map library to merge
const merged = await mergeSourceMaps(inputSourceMap, outputSourceMap);
return { js, sourceMap: merged };
}
```
### 8.4 Error Mapping Implementation
#### 8.4.1 Store Source Maps with Compartments
```typescript
// packages/runner/src/sandbox/compartment-manager.ts
interface PatternCompartment {
compartment: Compartment;
exports: Map;
sourceMap: SourceMap; // Add source map storage
sourceFiles: Map; // Original source for display
}
function createPatternCompartment(
compiled: JsScript,
runtimeGlobals: Record
): PatternCompartment {
// ... existing compartment creation ...
return {
compartment,
exports: frozenExports,
sourceMap: compiled.sourceMap,
sourceFiles: compiled.sourceFiles, // From compilation
};
}
```
#### 8.4.2 Error Mapping Utility
```typescript
// packages/runner/src/sandbox/error-mapping.ts
import { SourceMapConsumer } from 'source-map';
interface MappedFrame {
functionName: string;
fileName: string;
lineNumber: number;
columnNumber: number;
originalFunctionName?: string; // e.g., "computed callback"
isHoisted: boolean;
}
interface MappedError extends Error {
originalStack: string;
mappedStack: string;
mappedFrames: MappedFrame[];
patternId?: string;
}
export async function mapError(
error: Error,
sourceMap: SourceMap,
patternId: string
): Promise {
const consumer = await new SourceMapConsumer(sourceMap);
try {
const frames = parseStackTrace(error.stack);
const mappedFrames: MappedFrame[] = [];
for (const frame of frames) {
if (isPatternFrame(frame, patternId)) {
const original = consumer.originalPositionFor({
line: frame.lineNumber,
column: frame.columnNumber,
});
if (original.source) {
mappedFrames.push({
functionName: original.name || frame.functionName,
fileName: original.source,
lineNumber: original.line,
columnNumber: original.column,
originalFunctionName: getOriginalName(frame.functionName),
isHoisted: frame.functionName.startsWith('__'),
});
} else {
mappedFrames.push({ ...frame, isHoisted: false });
}
} else {
// Non-pattern frame, keep as-is
mappedFrames.push({ ...frame, isHoisted: false });
}
}
const mappedError = error as MappedError;
mappedError.originalStack = error.stack;
mappedError.mappedStack = formatMappedStack(mappedFrames);
mappedError.mappedFrames = mappedFrames;
mappedError.patternId = patternId;
mappedError.stack = mappedError.mappedStack;
return mappedError;
} finally {
consumer.destroy();
}
}
function getOriginalName(hoistedName: string): string | undefined {
// __computed_1 → "computed callback"
// __action_2 → "action callback"
// __derive_3 → "derive callback"
if (hoistedName.startsWith('__computed_')) return 'computed callback';
if (hoistedName.startsWith('__action_')) return 'action callback';
if (hoistedName.startsWith('__derive_')) return 'derive callback';
return undefined;
}
function formatMappedStack(frames: MappedFrame[]): string {
return frames.map(frame => {
let line = ` at ${frame.functionName} (${frame.fileName}:${frame.lineNumber}:${frame.columnNumber})`;
if (frame.isHoisted && frame.originalFunctionName) {
line += `\n └─ (originally: ${frame.originalFunctionName})`;
}
return line;
}).join('\n');
}
```
#### 8.4.3 Wrap Execution with Error Mapping
```typescript
// packages/runner/src/sandbox/execution-wrapper.ts
export async function executeWithErrorMapping(
fn: () => T,
patternCompartment: PatternCompartment,
patternId: string
): Promise {
try {
return fn();
} catch (error) {
if (error instanceof Error && patternCompartment.sourceMap) {
const mappedError = await mapError(
error,
patternCompartment.sourceMap,
patternId
);
throw mappedError;
}
throw error;
}
}
// Usage in runner
invoke(patternId: string, exportName: string, input: unknown): unknown {
const pattern = this.patternCompartments.get(patternId);
const exp = pattern.exports.get(exportName);
return executeWithErrorMapping(
() => exp.implementation(input),
pattern,
patternId
);
}
```
### 8.5 Debugging Experience
#### 8.5.1 Console Output
With proper error mapping, developers see:
```
TypeError: Cannot read property 'map' of undefined
at computed callback (MyPattern.tsx:23:42)
└─ (originally: computed callback)
at MyPattern (MyPattern.tsx:22:3)
at SandboxedRunner.invoke (compartment-manager.ts:89:5)
Pattern: my-pattern-id
Export: MyPattern
Original source (MyPattern.tsx:23):
22 │ export const MyPattern = pattern((props) => {
> 23 │ const doubled = computed(() => props.value.map(x => x * 2));
│ ^^^
24 │ return { doubled };
```
#### 8.5.2 Enhanced Error Display
```typescript
// packages/runner/src/sandbox/error-display.ts
export function formatErrorForDisplay(
error: MappedError,
sourceFiles: Map
): string {
const lines: string[] = [
`${error.name}: ${error.message}`,
'',
error.mappedStack,
];
// Add pattern context
if (error.patternId) {
lines.push('', `Pattern: ${error.patternId}`);
}
// Add source context if available
const topFrame = error.mappedFrames[0];
if (topFrame && sourceFiles.has(topFrame.fileName)) {
const source = sourceFiles.get(topFrame.fileName);
const context = extractSourceContext(
source,
topFrame.lineNumber,
topFrame.columnNumber
);
lines.push('', `Original source (${topFrame.fileName}:${topFrame.lineNumber}):`);
lines.push(context);
}
return lines.join('\n');
}
function extractSourceContext(
source: string,
line: number,
column: number,
contextLines: number = 1
): string {
const lines = source.split('\n');
const start = Math.max(0, line - 1 - contextLines);
const end = Math.min(lines.length, line + contextLines);
const result: string[] = [];
for (let i = start; i < end; i++) {
const lineNum = i + 1;
const prefix = lineNum === line ? '> ' : ' ';
const numStr = String(lineNum).padStart(3);
result.push(`${prefix}${numStr} │ ${lines[i]}`);
// Add column pointer for error line
if (lineNum === line) {
const pointer = ' '.repeat(column + 7) + '^^^';
result.push(` │ ${pointer}`);
}
}
return result.join('\n');
}
```
### 8.6 Layered Stack Trace Filtering
The key insight is that **pattern authors and runtime developers have different needs**:
- **Pattern authors** need to see their code, but runtime internals are noise
- **Runtime developers** need to see everything when debugging the runtime itself
#### 8.6.1 Frame Classification
```typescript
// packages/runner/src/sandbox/frame-classifier.ts
type FrameType = 'pattern' | 'runtime' | 'external';
interface ClassifiedFrame extends MappedFrame {
frameType: FrameType;
}
function classifyFrame(frame: MappedFrame, patternId: string): FrameType {
// Pattern code - always from the pattern's source files
if (isPatternSource(frame.fileName, patternId)) {
return 'pattern';
}
// Runtime code - our internal packages
if (isRuntimeSource(frame.fileName)) {
return 'runtime';
}
// External - third-party libraries, esm.sh imports
return 'external';
}
function isPatternSource(fileName: string, patternId: string): boolean {
// Pattern sources are in the virtual filesystem or have pattern markers
return fileName.includes(patternId) ||
fileName.endsWith('.tsx') ||
fileName.startsWith('/patterns/');
}
function isRuntimeSource(fileName: string): boolean {
return fileName.includes('packages/runner/') ||
fileName.includes('packages/common-builder/') ||
fileName.includes('packages/common-html/') ||
fileName.includes('compartment-manager') ||
fileName.includes('sandbox/');
}
```
#### 8.6.2 Filtered Stack Trace Output
**For Pattern Authors (default):**
```
TypeError: Cannot read property 'map' of undefined
at computed callback (MyPattern.tsx:23:42)
└─ props.value is undefined
at MyPattern (MyPattern.tsx:22:3)
... 3 runtime frames hidden (use --debug for full trace)
Pattern: my-pattern-id
Original source (MyPattern.tsx:23):
22 │ export const MyPattern = pattern((props) => {
> 23 │ const doubled = computed(() => props.value.map(x => x * 2));
│ ^^^
24 │ return { doubled };
```
**For Runtime Developers (debug mode):**
```
TypeError: Cannot read property 'map' of undefined
at computed callback (MyPattern.tsx:23:42)
└─ props.value is undefined
at MyPattern (MyPattern.tsx:22:3)
─── runtime frames ───
at FrozenExport.implementation (compartment-manager.ts:89:5)
at SandboxedRunner.invoke (runner.ts:1254:12)
at executeWithErrorMapping (execution-wrapper.ts:15:12)
at instantiateJavaScriptNode (runner.ts:1174:8)
─── end runtime frames ───
Pattern: my-pattern-id
```
#### 8.6.3 Implementation
```typescript
// packages/runner/src/sandbox/stack-filter.ts
interface StackFilterOptions {
showRuntimeFrames: boolean; // false for pattern authors, true for runtime devs
showExternalFrames: boolean; // usually true
maxPatternFrames: number; // limit depth, default unlimited
}
const DEFAULT_OPTIONS: StackFilterOptions = {
showRuntimeFrames: false,
showExternalFrames: true,
maxPatternFrames: Infinity,
};
export function filterStack(
frames: ClassifiedFrame[],
options: StackFilterOptions = DEFAULT_OPTIONS
): { visibleFrames: ClassifiedFrame[]; hiddenCount: number } {
const visibleFrames: ClassifiedFrame[] = [];
let hiddenCount = 0;
let patternFrameCount = 0;
for (const frame of frames) {
switch (frame.frameType) {
case 'pattern':
if (patternFrameCount < options.maxPatternFrames) {
visibleFrames.push(frame);
patternFrameCount++;
} else {
hiddenCount++;
}
break;
case 'runtime':
if (options.showRuntimeFrames) {
visibleFrames.push(frame);
} else {
hiddenCount++;
}
break;
case 'external':
if (options.showExternalFrames) {
visibleFrames.push(frame);
} else {
hiddenCount++;
}
break;
}
}
return { visibleFrames, hiddenCount };
}
export function formatFilteredStack(
frames: ClassifiedFrame[],
options: StackFilterOptions
): string {
const { visibleFrames, hiddenCount } = filterStack(frames, options);
const lines: string[] = [];
let inRuntimeSection = false;
for (const frame of visibleFrames) {
// Add section markers for runtime frames in debug mode
if (options.showRuntimeFrames) {
if (frame.frameType === 'runtime' && !inRuntimeSection) {
lines.push(' ─── runtime frames ───');
inRuntimeSection = true;
} else if (frame.frameType !== 'runtime' && inRuntimeSection) {
lines.push(' ─── end runtime frames ───');
inRuntimeSection = false;
}
}
lines.push(formatFrame(frame));
}
if (inRuntimeSection) {
lines.push(' ─── end runtime frames ───');
}
if (hiddenCount > 0 && !options.showRuntimeFrames) {
lines.push(` ... ${hiddenCount} runtime frames hidden (use --debug for full trace)`);
}
return lines.join('\n');
}
```
#### 8.6.4 Debug Mode Activation
```typescript
// packages/runner/src/sandbox/config.ts
export interface SandboxConfig {
// For pattern authors (default)
errorDisplay: 'pattern-only' | 'full';
// Environment detection
isRuntimeDeveloper: boolean;
}
// Auto-detect based on environment
export function detectConfig(): SandboxConfig {
return {
errorDisplay: process.env.COMMON_TOOLS_DEBUG === 'true'
? 'full'
: 'pattern-only',
isRuntimeDeveloper:
process.env.COMMON_TOOLS_DEBUG === 'true' ||
process.env.NODE_ENV === 'development' &&
isRunningFromSource(), // e.g., not from node_modules
};
}
function isRunningFromSource(): boolean {
// Check if we're running from the monorepo vs installed package
return __dirname.includes('/packages/runner/src/');
}
```
### 8.7 Configuration Summary
| Audience | `errorDisplay` | Runtime Frames | Source Context |
|----------|----------------|----------------|----------------|
| Pattern Author | `'pattern-only'` | Hidden | Pattern source shown |
| Runtime Developer | `'full'` | Visible (marked) | All source shown |
| Production (logging) | `'pattern-only'` | Hidden | Included in logs |
```typescript
// Pattern author sees clean errors focused on their code
CompartmentManager.configure({
errorDisplay: 'pattern-only',
});
// Runtime developer sees everything
CompartmentManager.configure({
errorDisplay: 'full',
});
// Or via environment:
// COMMON_TOOLS_DEBUG=true
```
### 8.8 Implementation Checklist
| Task | Priority | Files |
|------|----------|-------|
| Transformer source map generation | High | `ts-transformers/src/hoisting.ts` |
| Source map chaining in js-compiler | High | `js-compiler/typescript/compiler.ts` |
| Store source maps in PatternCompartment | High | `runner/src/sandbox/compartment-manager.ts` |
| Error mapping utility | High | `runner/src/sandbox/error-mapping.ts` |
| Execution wrapper with mapping | High | `runner/src/sandbox/execution-wrapper.ts` |
| Enhanced error display | Medium | `runner/src/sandbox/error-display.ts` |
| Source context extraction | Medium | `runner/src/sandbox/error-display.ts` |
| Configuration options | Low | `runner/src/sandbox/config.ts` |
---
## 9. Implementation Plan
### Phase 1: Transformer Enhancements
#### 1.1 Module-Scope Validation (Priority: High)
Add `ModuleScopeValidationTransformer`:
```typescript
class ModuleScopeValidationTransformer {
// Allowlist of permitted module-scope calls
private allowedCalls = new Set([
'pattern', 'pattern', 'lift', 'handler',
'Object.freeze', 'harden'
]);
visitCallExpression(node: ts.CallExpression): void {
if (this.isModuleScope(node)) {
const callee = this.getCalleeName(node);
if (!this.allowedCalls.has(callee)) {
this.reportError(node, `Call to ${callee} not allowed at module scope`);
}
}
}
visitVariableDeclaration(node: ts.VariableDeclaration): void {
if (this.isModuleScope(node) && node.kind !== ts.SyntaxKind.ConstKeyword) {
this.reportError(node, 'Only const declarations allowed at module scope');
}
}
}
```
**Files to modify:**
- `packages/ts-transformers/src/index.ts` - Add new transformer to pipeline
- New file: `packages/ts-transformers/src/module-scope-validation.ts`
#### 1.2 Hoist Computed/Action/Derive (Priority: High)
Modify `ComputedTransformer` and `ClosureTransformer`:
```typescript
class HoistingTransformer {
private hoistedDeclarations: ts.Statement[] = [];
private counter = 0;
visitComputedCall(node: ts.CallExpression): ts.Expression {
const fn = node.arguments[0];
const name = `__computed_${this.counter++}`;
// Create hoisted lift
const hoisted = ts.factory.createVariableStatement(
undefined,
ts.factory.createVariableDeclarationList([
ts.factory.createVariableDeclaration(
name,
undefined,
undefined,
ts.factory.createCallExpression(
ts.factory.createIdentifier('lift'),
[...typeArgs],
[fn]
)
)
], ts.NodeFlags.Const)
);
this.hoistedDeclarations.push(hoisted);
// Return call to hoisted lift
return ts.factory.createCallExpression(
ts.factory.createIdentifier(name),
undefined,
[capturedInputs]
);
}
}
```
**Files to modify:**
- `packages/ts-transformers/src/computed.ts`
- `packages/ts-transformers/src/closure.ts`
- New file: `packages/ts-transformers/src/hoisting.ts`
#### 1.3 Export Name Annotation (Priority: Medium)
Add annotation to all builder calls:
```typescript
class ExportNameAnnotationTransformer {
visitExportDeclaration(node: ts.ExportDeclaration): ts.Node {
// For: export const MyPattern = pattern(...);
// Add: MyPattern.__exportName = "MyPattern";
const name = this.getExportName(node);
const annotation = ts.factory.createExpressionStatement(
ts.factory.createAssignment(
ts.factory.createPropertyAccessExpression(
ts.factory.createIdentifier(name),
'__exportName'
),
ts.factory.createStringLiteral(name)
)
);
return [node, annotation];
}
}
```
**Files to modify:**
- New file: `packages/ts-transformers/src/export-annotation.ts`
### Phase 2: SES Integration
#### 2.1 Add SES Dependency
```bash
npm install ses
```
**Files to modify:**
- `packages/runner/package.json`
#### 2.2 Create Compartment Manager
New module for managing pattern Compartments:
```typescript
// packages/runner/src/sandbox/compartment-manager.ts
import 'ses';
export class CompartmentManager {
private static lockdownApplied = false;
private patternCompartments = new Map();
static applyLockdown(): void {
if (this.lockdownApplied) return;
lockdown({
errorTaming: 'unsafe',
stackFiltering: 'verbose',
overrideTaming: 'moderate',
consoleTaming: 'unsafe',
});
this.lockdownApplied = true;
}
loadPattern(id: string, compiledJS: string): PatternCompartment { ... }
getExport(patternId: string, name: string): FrozenExport { ... }
evaluateString(code: string): Function { ... }
}
```
**Files to create:**
- `packages/runner/src/sandbox/compartment-manager.ts`
- `packages/runner/src/sandbox/pattern-compartment.ts`
- `packages/runner/src/sandbox/types.ts`
#### 2.3 Create Runtime Globals Provider
Define the frozen globals available in pattern Compartments:
```typescript
// packages/runner/src/sandbox/runtime-globals.ts
export function createRuntimeGlobals(): Record {
return harden({
// Builder functions
pattern: createPatternBuilder(),
pattern: createPatternBuilder(),
lift: createLiftBuilder(),
handler: createHandlerBuilder(),
derive: createDeriveBuilder(),
// Cell/reactive
Cell,
cell,
// UI
h,
// Standard globals (frozen)
console: createSandboxedConsole(),
JSON,
Math,
Object: {
keys: Object.keys,
values: Object.values,
entries: Object.entries,
freeze: Object.freeze,
// ... allowlisted methods only
},
Array: {
isArray: Array.isArray,
from: Array.from,
// ... allowlisted methods only
},
});
}
```
**Files to create:**
- `packages/runner/src/sandbox/runtime-globals.ts`
- `packages/runner/src/sandbox/sandboxed-console.ts`
### Phase 3: Runner Integration
#### 3.1 Modify instantiateJavaScriptNode
Replace direct eval with Compartment invocation:
```typescript
// packages/runner/src/runner.ts
private instantiateJavaScriptNode(
tx: IExtendedStorageTransaction,
module: JavaScriptModuleDefinition,
...
): void {
let fn: Function;
if (typeof module.implementation === "string") {
// Check if this is a frozen export from a loaded pattern
if (module.patternId && module.exportName) {
const exp = this.compartmentManager.getExport(
module.patternId,
module.exportName
);
// Verify it's frozen and module-scope defined
verifyFrozen(exp, module.exportName);
fn = exp.implementation;
} else {
// Fallback: evaluate in fresh Compartment
fn = this.compartmentManager.evaluateString(module.implementation);
}
} else {
fn = module.implementation;
}
// ... rest of existing logic ...
}
```
**Files to modify:**
- `packages/runner/src/runner.ts`
#### 3.2 Remove UnsafeEvalIsolate Usage
Replace `harness.getInvocation()` with Compartment-based evaluation:
```typescript
// Before
fn = this.runtime.harness.getInvocation(module.implementation);
// After
fn = this.compartmentManager.evaluateString(module.implementation);
```
**Files to modify:**
- `packages/runner/src/harness/engine.ts` (deprecate or remove)
- `packages/runner/src/harness/eval-runtime.ts` (deprecate or remove)
### Phase 4: Dynamic Import Support
#### 4.1 Implement Import Hooks
```typescript
// packages/runner/src/sandbox/import-hooks.ts
export function createImportHooks(
esmCache: Map
): ImportHooks {
let invocationCounter = 0;
return {
resolveHook(specifier: string, referrer: string): string {
if (isEsmShUrl(specifier)) {
return `${specifier}#__inv_${invocationCounter++}`;
}
return resolveStandard(specifier, referrer);
},
async importHook(specifier: string): Promise {
const url = stripInvocationSuffix(specifier);
let source = esmCache.get(url);
if (!source) {
source = await fetchAndCache(url, esmCache);
}
return new StaticModuleRecord(source, specifier);
},
};
}
```
**Files to create:**
- `packages/runner/src/sandbox/import-hooks.ts`
- `packages/runner/src/sandbox/esm-cache.ts`
#### 4.2 Integrate with Compartment
```typescript
// packages/runner/src/sandbox/dynamic-import-compartment.ts
export async function createDynamicImportCompartment(
base: Compartment,
esmCache: Map
): Promise {
const hooks = createImportHooks(esmCache);
return new Compartment(
base.globalThis,
{},
{
resolveHook: hooks.resolveHook,
importHook: hooks.importHook,
}
);
}
```
**Files to create:**
- `packages/runner/src/sandbox/dynamic-import-compartment.ts`
### Phase 5: Testing & Hardening
#### 5.1 Security Tests
```typescript
// packages/runner/test/sandbox/security.test.ts
describe('SES Sandbox Security', () => {
it('prevents closure state leakage between invocations', async () => {
const pattern = `
let leaked;
export const TestPattern = pattern((props) => {
leaked = props.secret;
return { getter: () => leaked };
});
`;
// This should fail at load time (module-scope mutation)
await expect(loadPattern(pattern)).rejects.toThrow();
});
it('prevents access to global objects', async () => {
const pattern = `
export const TestPattern = pattern(() => {
return { hasProcess: typeof process !== 'undefined' };
});
`;
const result = await invokePattern(pattern, {});
expect(result.hasProcess).toBe(false);
});
it('isolates dynamic imports between invocations', async () => {
const pattern = `
export const TestPattern = pattern(async () => {
const mod = await import('https://esm.sh/stateful-module');
mod.increment();
return { count: mod.getCount() };
});
`;
const result1 = await invokePattern(pattern, {});
const result2 = await invokePattern(pattern, {});
// Each should start fresh
expect(result1.count).toBe(1);
expect(result2.count).toBe(1); // NOT 2!
});
it('freezes all pattern exports', async () => {
const pattern = `
export const myLift = lift((x) => x * 2);
`;
const compartment = await loadPattern(pattern);
const exp = compartment.exports.get('myLift');
expect(Object.isFrozen(exp)).toBe(true);
expect(Object.isFrozen(exp.implementation)).toBe(true);
});
});
```
**Files to create:**
- `packages/runner/test/sandbox/security.test.ts`
- `packages/runner/test/sandbox/compartment.test.ts`
- `packages/runner/test/sandbox/import-hooks.test.ts`
#### 5.2 Performance Tests
```typescript
// packages/runner/test/sandbox/performance.test.ts
describe('SES Sandbox Performance', () => {
it('reuses Compartment for multiple invocations', async () => {
const pattern = loadPattern(source);
const start = performance.now();
for (let i = 0; i < 1000; i++) {
await invokePattern(pattern, { value: i });
}
const elapsed = performance.now() - start;
// Should be fast since no Compartment creation per invocation
expect(elapsed).toBeLessThan(1000); // < 1ms per invocation
});
});
```
---
## 10. Migration Guide
### 10.1 Pattern Author Changes
Most patterns will work without changes. The following patterns require updates:
#### Patterns with module-scope side effects
**Before (breaks):**
```typescript
const startTime = Date.now(); // Side effect at module scope
```
**After:**
```typescript
// Move to a lift if needed
const getStartTime = lift(() => Date.now());
```
#### Patterns with mutable module-scope state
**Before (breaks):**
```typescript
let counter = 0;
export const MyPattern = pattern(() => {
counter++;
return { count: counter };
});
```
**After:**
```typescript
// Use Cell for state
export const MyPattern = pattern(() => {
const counter = cell(0);
const increment = handler(() => counter.set(counter.get() + 1));
return { count: counter, increment };
});
```
### 10.2 Runtime API Changes
```typescript
// Before
const runner = new Runner(runtime);
runner.start(pattern, inputs);
// After (if explicit lockdown control needed)
CompartmentManager.applyLockdown(); // Call once at startup
const runner = new Runner(runtime);
runner.start(pattern, inputs);
```
---
## 11. Security Considerations
### 11.1 Threat Model
| Threat | Mitigation |
|--------|------------|
| Arbitrary code execution | SES Compartment isolation |
| Global pollution | Frozen intrinsics, controlled globals |
| Prototype pollution | Frozen prototypes (SES default) |
| Closure-based data leakage | No surviving closures, hoisted frozen implementations |
| State leakage via modules | Fresh Compartments for dynamic imports |
| Resource exhaustion | Future: Add CPU/memory limits (not in this spec) |
| Network access | Future: Control fetch in globals (not in this spec) |
### 11.2 Known Limitations
1. **No CPU limits**: Infinite loops will still hang. Future work: Integrate with QuickJS for CPU limits or use Web Workers with timeouts.
2. **No memory limits**: Memory exhaustion possible. Future work: Monitor heap usage.
3. **No network restrictions**: `fetch` is not blocked. Future work: Proxy `fetch` with allowlist.
### 11.3 Escape Hatch Analysis
Potential escape routes and their status:
| Vector | Status | Notes |
|--------|--------|-------|
| `eval()` | Blocked | SES removes `eval` from Compartment globals |
| `Function()` | Blocked | SES removes `Function` constructor |
| `import()` | Controlled | Via import hooks |
| Prototype access | Blocked | Frozen prototypes |
| `globalThis` | Controlled | Custom Compartment globals |
| `__proto__` | Blocked | Frozen Object.prototype |
| `constructor` | Blocked | Frozen constructors |
---
## 12. Appendix
### A. SES Package Selection
**Recommended**: `ses` npm package (official from Agoric)
**Alternatives considered**:
- `@aspect-labs/ses` - Fork with minor fixes
- `lavamoat` - Higher-level, more opinionated
- QuickJS - Different approach (separate runtime)
### B. AMD Loader Compatibility
The existing AMD loader in `js-compiler` is compatible with SES Compartments. The loader is already:
- Self-contained (no global access)
- Pure (no side effects beyond module registration)
- Configurable (accepts runtime dependencies)
### C. Glossary
- **Compartment**: SES isolation boundary with its own global object
- **Harden**: Deep freeze an object graph
- **Lockdown**: Initialize SES, freeze all intrinsics
- **StaticModuleRecord**: SES's representation of an ES module
- **Import hooks**: Callbacks for resolving and loading modules
---
## 13. References
1. [SES (Secure ECMAScript)](https://github.com/endojs/endo/tree/master/packages/ses)
2. [Hardened JavaScript](https://hardenedjs.org/)
3. [Compartment API](https://github.com/tc39/proposal-compartments)
4. [Common Tools Pattern Documentation](../common/INTRODUCTION.md)
5. [ts-transformers Package](../../packages/ts-transformers/)
6. [js-compiler Package](../../packages/js-compiler/)