| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418 |
- /**
- * CodeGraph Utilities
- *
- * Common utility functions for memory management, concurrency, batching,
- * and security validation.
- *
- * @module utils
- *
- * @example
- * ```typescript
- * import { Mutex, processInBatches, MemoryMonitor, validatePathWithinRoot } from 'codegraph';
- *
- * // Use mutex for concurrent safety
- * const mutex = new Mutex();
- * await mutex.withLock(async () => {
- * await performCriticalOperation();
- * });
- *
- * // Process items in batches to manage memory
- * const results = await processInBatches(items, 100, async (item) => {
- * return await processItem(item);
- * });
- *
- * // Monitor memory usage
- * const monitor = new MemoryMonitor(512, (usage) => {
- * console.warn(`Memory usage exceeded 512MB: ${usage / 1024 / 1024}MB`);
- * });
- * monitor.start();
- * ```
- */
- import * as path from 'path';
- import * as fs from 'fs';
- // ============================================================
- // SECURITY UTILITIES
- // ============================================================
- /**
- * Validate that a resolved file path stays within the project root.
- * Prevents path traversal attacks (e.g. node.filePath = "../../etc/passwd").
- *
- * @param projectRoot - The project root directory
- * @param filePath - The relative file path to validate
- * @returns The resolved absolute path, or null if it escapes the root
- */
- export function validatePathWithinRoot(projectRoot: string, filePath: string): string | null {
- const resolved = path.resolve(projectRoot, filePath);
- const normalizedRoot = path.resolve(projectRoot);
- if (!resolved.startsWith(normalizedRoot + path.sep) && resolved !== normalizedRoot) {
- return null;
- }
- return resolved;
- }
- /**
- * Safely parse JSON with a fallback value.
- * Prevents crashes from corrupted database metadata.
- */
- export function safeJsonParse<T>(value: string, fallback: T): T {
- try {
- return JSON.parse(value);
- } catch {
- return fallback;
- }
- }
- /**
- * Clamp a numeric value to a range.
- * Used to enforce sane limits on MCP tool inputs.
- */
- export function clamp(value: number, min: number, max: number): number {
- return Math.max(min, Math.min(max, value));
- }
- /**
- * Normalize a file path to use forward slashes.
- * Fixes Windows backslash paths so glob matching works consistently.
- */
- export function normalizePath(filePath: string): string {
- return filePath.replace(/\\/g, '/');
- }
- /**
- * Cross-process file lock using lock files.
- * Prevents concurrent database writes from CLI, MCP server, and git hooks.
- */
- export class FileLock {
- private lockPath: string;
- private acquired = false;
- constructor(resourcePath: string) {
- this.lockPath = resourcePath + '.lock';
- }
- /**
- * Acquire the file lock. Waits up to timeoutMs for the lock.
- * Cleans up stale locks older than staleLockMs.
- */
- async acquire(timeoutMs: number = 10000, staleLockMs: number = 30000): Promise<boolean> {
- const start = Date.now();
- while (Date.now() - start < timeoutMs) {
- try {
- // Try to create lock file exclusively
- fs.writeFileSync(this.lockPath, String(process.pid), { flag: 'wx' });
- this.acquired = true;
- return true;
- } catch {
- // Lock file exists - check if stale
- try {
- const stat = fs.statSync(this.lockPath);
- if (Date.now() - stat.mtimeMs > staleLockMs) {
- // Stale lock - remove and retry
- fs.unlinkSync(this.lockPath);
- continue;
- }
- } catch {
- // Lock file disappeared between check and stat - retry
- continue;
- }
- // Wait and retry
- await new Promise(resolve => setTimeout(resolve, 100));
- }
- }
- return false;
- }
- /**
- * Release the file lock
- */
- release(): void {
- if (this.acquired) {
- try {
- fs.unlinkSync(this.lockPath);
- } catch {
- // Lock file already removed - that's fine
- }
- this.acquired = false;
- }
- }
- }
- /**
- * Process items in batches to manage memory
- *
- * @param items - Array of items to process
- * @param batchSize - Number of items per batch
- * @param processor - Function to process each item
- * @param onBatchComplete - Optional callback after each batch
- * @returns Array of results
- */
- export async function processInBatches<T, R>(
- items: T[],
- batchSize: number,
- processor: (item: T, index: number) => Promise<R>,
- onBatchComplete?: (completed: number, total: number) => void
- ): Promise<R[]> {
- const results: R[] = [];
- for (let i = 0; i < items.length; i += batchSize) {
- const batch = items.slice(i, Math.min(i + batchSize, items.length));
- const batchResults = await Promise.all(
- batch.map((item, idx) => processor(item, i + idx))
- );
- results.push(...batchResults);
- if (onBatchComplete) {
- onBatchComplete(Math.min(i + batchSize, items.length), items.length);
- }
- // Allow GC between batches
- if (global.gc) {
- global.gc();
- }
- }
- return results;
- }
- /**
- * Simple mutex lock for preventing concurrent operations
- */
- export class Mutex {
- private locked = false;
- private waitQueue: Array<() => void> = [];
- /**
- * Acquire the lock
- *
- * @returns A release function to call when done
- */
- async acquire(): Promise<() => void> {
- while (this.locked) {
- await new Promise<void>((resolve) => {
- this.waitQueue.push(resolve);
- });
- }
- this.locked = true;
- return () => {
- this.locked = false;
- const next = this.waitQueue.shift();
- if (next) {
- next();
- }
- };
- }
- /**
- * Execute a function while holding the lock
- */
- async withLock<T>(fn: () => Promise<T> | T): Promise<T> {
- const release = await this.acquire();
- try {
- return await fn();
- } finally {
- release();
- }
- }
- /**
- * Check if the lock is currently held
- */
- isLocked(): boolean {
- return this.locked;
- }
- }
- /**
- * Chunked file reader for large files
- *
- * Reads a file in chunks to avoid loading entire file into memory.
- */
- export async function* readFileInChunks(
- filePath: string,
- chunkSize: number = 64 * 1024
- ): AsyncGenerator<string, void, undefined> {
- const fs = await import('fs');
- const fd = fs.openSync(filePath, 'r');
- const buffer = Buffer.alloc(chunkSize);
- try {
- let bytesRead: number;
- while ((bytesRead = fs.readSync(fd, buffer, 0, chunkSize, null)) > 0) {
- yield buffer.toString('utf-8', 0, bytesRead);
- }
- } finally {
- fs.closeSync(fd);
- }
- }
- /**
- * Debounce a function
- *
- * @param fn - Function to debounce
- * @param delay - Delay in milliseconds
- * @returns Debounced function
- */
- export function debounce<T extends (...args: unknown[]) => unknown>(
- fn: T,
- delay: number
- ): (...args: Parameters<T>) => void {
- let timeoutId: ReturnType<typeof setTimeout> | null = null;
- return (...args: Parameters<T>) => {
- if (timeoutId) {
- clearTimeout(timeoutId);
- }
- timeoutId = setTimeout(() => {
- fn(...args);
- timeoutId = null;
- }, delay);
- };
- }
- /**
- * Throttle a function
- *
- * @param fn - Function to throttle
- * @param limit - Minimum time between calls in milliseconds
- * @returns Throttled function
- */
- export function throttle<T extends (...args: unknown[]) => unknown>(
- fn: T,
- limit: number
- ): (...args: Parameters<T>) => void {
- let lastCall = 0;
- let timeoutId: ReturnType<typeof setTimeout> | null = null;
- return (...args: Parameters<T>) => {
- const now = Date.now();
- const remaining = limit - (now - lastCall);
- if (remaining <= 0) {
- if (timeoutId) {
- clearTimeout(timeoutId);
- timeoutId = null;
- }
- lastCall = now;
- fn(...args);
- } else if (!timeoutId) {
- timeoutId = setTimeout(() => {
- lastCall = Date.now();
- timeoutId = null;
- fn(...args);
- }, remaining);
- }
- };
- }
- /**
- * Estimate memory usage of an object (rough approximation)
- *
- * @param obj - Object to measure
- * @returns Approximate size in bytes
- */
- export function estimateSize(obj: unknown): number {
- const seen = new WeakSet();
- function sizeOf(value: unknown): number {
- if (value === null || value === undefined) {
- return 0;
- }
- switch (typeof value) {
- case 'boolean':
- return 4;
- case 'number':
- return 8;
- case 'string':
- return 2 * (value as string).length;
- case 'object':
- if (seen.has(value as object)) {
- return 0;
- }
- seen.add(value as object);
- if (Array.isArray(value)) {
- return value.reduce((acc: number, item) => acc + sizeOf(item), 0);
- }
- return Object.entries(value as object).reduce(
- (acc, [key, val]) => acc + sizeOf(key) + sizeOf(val),
- 0
- );
- default:
- return 0;
- }
- }
- return sizeOf(obj);
- }
- /**
- * Memory monitor for tracking usage during operations
- */
- export class MemoryMonitor {
- private checkInterval: ReturnType<typeof setInterval> | null = null;
- private peakUsage = 0;
- private threshold: number;
- private onThresholdExceeded?: (usage: number) => void;
- constructor(
- thresholdMB: number = 500,
- onThresholdExceeded?: (usage: number) => void
- ) {
- this.threshold = thresholdMB * 1024 * 1024;
- this.onThresholdExceeded = onThresholdExceeded;
- }
- /**
- * Start monitoring memory usage
- */
- start(intervalMs: number = 1000): void {
- this.stop();
- this.peakUsage = 0;
- this.checkInterval = setInterval(() => {
- const usage = process.memoryUsage().heapUsed;
- if (usage > this.peakUsage) {
- this.peakUsage = usage;
- }
- if (usage > this.threshold && this.onThresholdExceeded) {
- this.onThresholdExceeded(usage);
- }
- }, intervalMs);
- }
- /**
- * Stop monitoring
- */
- stop(): void {
- if (this.checkInterval) {
- clearInterval(this.checkInterval);
- this.checkInterval = null;
- }
- }
- /**
- * Get peak memory usage in bytes
- */
- getPeakUsage(): number {
- return this.peakUsage;
- }
- /**
- * Get current memory usage in bytes
- */
- getCurrentUsage(): number {
- return process.memoryUsage().heapUsed;
- }
- }
|