Core
Registry of named callables (Registry + Command pattern).
The registry decouples “what to run” (a string name inside a JSON action list)
from “how to run it” (a Python callable). Executors delegate name resolution
to an ActionRegistry, which keeps look-up O(1) and lets plugins add
commands at runtime without touching the executor class.
- class automation_file.core.action_registry.ActionRegistry(initial=None)[source]
Bases:
objectMapping of action name -> callable.
- Parameters:
initial (Mapping[str, Command] | None)
- property event_dict: dict[str, Callable[[...], Any]]
Backwards-compatible view used by older
package_managerstyle code.
- automation_file.core.action_registry.build_default_registry()[source]
Return a registry pre-populated with every built-in
FA_*action.After the built-ins are registered, any third-party package advertising an
automation_file.actionsentry point is loaded so its commands land in the same registry. Plugins may override built-in names.- Return type:
Callback executor — runs a trigger, then a callback.
Implements the “do X then do Y” flow many automation JSON files want. The
registry is shared with ActionExecutor, so adding a command to one
adds it to the other.
- class automation_file.core.callback_executor.CallbackExecutor(registry)[source]
Bases:
objectInvoke
trigger(**kwargs)thencallback(*args | **kwargs).- Parameters:
registry (ActionRegistry)
Dynamic plugin registration into an ActionRegistry.
PackageLoader imports an external package by name and registers every
top-level function / class / builtin under the key "<package>_<member>".
- class automation_file.core.package_loader.PackageLoader(registry)[source]
Bases:
objectLoad packages lazily and register their public callables.
- Parameters:
registry (ActionRegistry)
- add_package_to_executor(package)[source]
Register every function / class / builtin from
package.Returns the number of commands that were registered.
- load(package)[source]
Import
packageonce and return the module (cached).- Parameters:
package (str)
- Return type:
ModuleType | None
JSON persistence for action lists.
Reads/writes are serialised through a module-level lock so concurrent callers cannot interleave writes against the same file.
- automation_file.core.json_store.read_action_json(json_file_path)[source]
Return the parsed JSON content at
json_file_path.
- automation_file.core.json_store.write_action_json(json_save_path, action_json)[source]
Write
action_jsontojson_save_pathas pretty UTF-8 JSON.
Retry helper for transient network failures.
retry_on_transient is a small wrapper around exponential back-off. It is
intentionally dependency-free so that modules which do not actually use
requests or googleapiclient can import it without pulling those in.
- automation_file.core.retry.retry_on_transient(max_attempts=3, backoff_base=0.5, backoff_cap=8.0, retriable=(<class 'ConnectionError'>, <class 'TimeoutError'>, <class 'OSError'>))[source]
Return a decorator that retries
retriableexceptions with back-off.On the final failure raises
RetryExhaustedExceptionchained to the underlying error so callers can still inspect the cause.
Per-action quota enforcement.
Quota bundles a maximum byte size and maximum duration. Callers use
Quota.check_size(bytes) before an I/O-heavy action and wrap the action in
with quota.time_budget(label): to bound wall-clock time.
- class automation_file.core.quota.Quota(max_bytes=0, max_seconds=0.0)[source]
Bases:
objectBundle of per-action limits.
max_bytes<= 0 means no size cap;max_seconds<= 0 means no time cap. Defaults allow callers to share oneQuotainstance across many actions with fine-grained overrides at each call site.- time_budget(label='action')[source]
Context manager that raises if the enclosed block runs past the cap.
- wraps(label, size_fn=None)[source]
Return a decorator that enforces the time budget around
func.If
size_fnis provided it is called with the function’s return value to derive a byte count forcheck_size().- Parameters:
label (str)
Token-bucket rate limiter.
RateLimiter refills at rate tokens/second up to a burst capacity.
Callers acquire N tokens before issuing a protected call; when empty, the
limiter either blocks (up to timeout) or raises
RateLimitExceededException.
- class automation_file.core.rate_limit.RateLimiter(rate, burst=None)[source]
Bases:
objectThread-safe token bucket.
- acquire(tokens=1.0, timeout=None)[source]
Block until
tokensare available.Raises
RateLimitExceededExceptioniftimeoutelapses first.timeout=Nonewaits indefinitely;timeout=0fails immediately.
Three-state circuit breaker.
States: CLOSED (normal), OPEN (short-circuit after failure_threshold
consecutive failures), HALF_OPEN (trial one call after recovery_timeout
seconds; one success closes, one failure re-opens). Failures are counted
only for exceptions in retriable — internal errors surface as-is.
- class automation_file.core.circuit_breaker.CircuitBreaker(failure_threshold=5, recovery_timeout=30.0, retriable=(<class 'Exception'>, ), name='circuit')[source]
Bases:
objectOpen-close-half-open breaker.
failure_threshold— consecutive failures that trip the breaker.recovery_timeout— seconds spent in OPEN before transitioning to HALF_OPEN.retriable— exception types counted as failures; other exceptions pass through.- Parameters:
Cross-platform advisory file lock.
Uses fcntl.flock on POSIX and msvcrt.locking on Windows, so two processes
can serialise on a well-known lock path. Locks are exclusive (writer-style);
shared locks are not supported because msvcrt cannot express them portably.
- class automation_file.core.file_lock.FileLock(path, timeout=None)[source]
Bases:
objectAdvisory exclusive lock on a sidecar lock file.
pathis the lock file itself — typically<resource>.locknext to the protected resource.timeoutis the maximum seconds to wait when acquiring;Nonewaits indefinitely,0fails immediately.- Parameters:
path (str | os.PathLike[str])
timeout (float | None)
SQLite-backed named lock for multi-process / multi-host coordination.
Unlike automation_file.core.file_lock.FileLock which locks a single
file descriptor, SQLiteLock persists named leases in a shared SQLite
database. Any process that can open the database can participate. Leases carry
an optional TTL so crashed owners eventually free the slot.
- class automation_file.core.sqlite_lock.SQLiteLock(db_path, name, timeout=None, ttl=None)[source]
Bases:
objectNamed lease stored in SQLite.
db_pathis the SQLite file — callers sharing a lock must point at the same file.nameis the lock identity.ttl(seconds) lets a crashed owner’s lease expire;Nonemeans the lease is held until explicit release.timeoutbounds acquisition wait.
Persistent SQLite-backed queue of action payloads.
Producers call ActionQueue.enqueue() to durably store a JSON action list;
consumers pull with dequeue() (marking the row inflight) and finalise
with ack() or nack(). The queue survives process restarts — all
state lives in the SQLite file.
- class automation_file.core.action_queue.ActionQueue(db_path)[source]
Bases:
objectDurable FIFO / priority queue for JSON action payloads.
- Parameters:
db_path (str | os.PathLike[str])
- ack(item_id)[source]
Finalise a claimed row as processed.
- Parameters:
item_id (int)
- Return type:
None
- dequeue()[source]
Claim the next ready row; returns
Noneif the queue is empty.- Return type:
QueueItem | None
- enqueue(action, priority=0, run_at=None)[source]
Persist
actionfor later dispatch. Returns the row id.
- class automation_file.core.action_queue.QueueItem(id, action, attempts, enqueued_at)[source]
Bases:
objectA claimed queue row returned by
ActionQueue.dequeue().
SHA-256 content-addressable store.
ContentStore ingests files or byte blobs and keys them by the hex
digest of their contents. A two-character fanout directory keeps any single
directory small: <root>/ab/abcdef…. Identical inputs map to the same blob —
callers get deduplication for free.
- class automation_file.core.content_store.ContentStore(root)[source]
Bases:
objectFilesystem-backed CAS under
root.- Parameters:
root (str | os.PathLike[str])
Transfer progress + cancellation primitives.
Long-running transfers (HTTP downloads, S3 uploads/downloads, …) accept a
named handle from the shared progress_registry. The registry keeps a
ProgressReporter (bytes transferred, optional total) and a
CancellationToken per name so the GUI or a JSON action can observe
progress or cancel mid-flight.
Instrumentation is opt-in: callers pass progress_name="<label>" to enable
tracking. When omitted, transfers run exactly as before with zero overhead
beyond one attribute lookup.
- class automation_file.core.progress.CancellationToken[source]
Bases:
objectThread-safe boolean flag, pollable from worker threads.
- exception automation_file.core.progress.CancelledException[source]
Bases:
FileAutomationExceptionRaised when a cancellable operation is asked to stop mid-flight.
- class automation_file.core.progress.ProgressRegistry[source]
Bases:
objectNamed handles so JSON actions / the GUI can address ongoing transfers.
- lookup(name)[source]
- Parameters:
name (str)
- Return type:
tuple[ProgressReporter, CancellationToken] | None
- class automation_file.core.progress.ProgressReporter(name, total=None, transferred=0, status='running', started_at=<factory>, finished_at=None, _lock=<factory>)[source]
Bases:
objectTracks bytes transferred for one named operation.
- Parameters:
- automation_file.core.progress.progress_cancel(name)[source]
Cancel the named transfer. Returns
Falseif no such handle.
- automation_file.core.progress.progress_clear()[source]
Drop every finished transfer from the registry.
- Return type:
- automation_file.core.progress.register_progress_ops(registry)[source]
Wire
FA_progress_*actions into anActionRegistry.- Parameters:
registry (Any)
- Return type:
None
File checksum + integrity verification helpers.
Streaming hashes so multi-GB files don’t blow up memory. The hash algorithm
is whatever hashlib.new() understands (sha256, sha1, md5,
blake2b, …). file_checksum() returns the hex digest;
verify_checksum() does a constant-time compare against the expected
value so callers can’t leak timing on the check.
- exception automation_file.core.checksum.ChecksumMismatchException[source]
Bases:
FileAutomationExceptionRaised when a computed digest does not match the expected value.
- automation_file.core.checksum.file_checksum(path, algorithm='sha256', chunk_size=1048576)[source]
Return the hex digest of
pathunderalgorithm.Reads the file in
chunk_sizeblocks so the memory cost is bounded regardless of file size. RaisesFileNotExistsExceptionwhen the path is missing andValueErrorfor unknown algorithms.
- automation_file.core.checksum.verify_checksum(path, expected, algorithm='sha256', chunk_size=1048576)[source]
Return True iff
path’s digest matchesexpected(case-insensitive).Uses
hmac.compare_digest()so the match check is constant-time.
Directory-tree manifests — JSON snapshot of every file’s checksum.
A manifest is a simple JSON document recording every file under a root, its size, and a streaming digest (SHA-256 by default). Two operations:
write_manifest(root, manifest_path) # snapshot now
verify_manifest(root, manifest_path) # verify the tree still matches
Use cases: release-artifact verification, backup integrity checks, detecting tampering on a sync target, or pre-flight checks before a rename / move.
The document shape is intentionally small and human-readable:
{
"version": 1,
"algorithm": "sha256",
"root": "/abs/path/at/snapshot/time",
"created_at": "2026-04-21T10:15:30+00:00",
"files": {
"a.txt": {"size": 3, "checksum": "..."},
"nested/b.txt": {"size": 1_024, "checksum": "..."}
}
}
Paths in the files mapping are POSIX-style (forward-slash) relative
paths so the manifest round-trips across Windows and Unix.
- exception automation_file.core.manifest.ManifestException[source]
Bases:
FileAutomationExceptionRaised for invalid manifest documents or unreadable manifest paths.
- automation_file.core.manifest.verify_manifest(root, manifest_path)[source]
Verify every file recorded in
manifest_pathstill matches underroot.Returns a summary dict:
{ "matched": ["a.txt"], "missing": ["gone.txt"], "modified": ["changed.txt"], "extra": ["new.txt"], # present under root, not in manifest "ok": False, }
okis True iffmissingandmodifiedare both empty (extras are reported but do not fail verification — mirrorsync_dir’s default non-deleting posture).
- automation_file.core.manifest.write_manifest(root, manifest_path, *, algorithm='sha256')[source]
Write a manifest for every file under
root. Returns the manifest dict.
SQLite-backed audit log for executed actions.
AuditLog(db_path) opens (or creates) a single-table SQLite database and
appends one row per action execution. Rows carry the timestamp, action name,
a JSON-encoded snapshot of the payload, the result / error repr, and the
duration in milliseconds.
Writes use a short-lived connection per call (check_same_thread=False
semantics) so the log is safe to share between background worker threads
and the scheduler. Readers call AuditLog.recent() to pull the most
recent N rows.
The module deliberately avoids buffering / background queues: every row is
persisted synchronously with an INSERT inside a with connect(..) so
a crash at most loses the currently-executing action.
- exception automation_file.core.audit.AuditException[source]
Bases:
FileAutomationExceptionRaised when the audit log cannot be opened or written.
- class automation_file.core.audit.AuditLog(db_path)[source]
Bases:
objectSynchronous SQLite audit log.
- Parameters:
db_path (str | Path)
- purge(older_than_seconds)[source]
Delete rows older than
older_than_secondsand return the row count.
Opt-in variable substitution for action list payloads.
When execute_action(..., substitute=True) is used, every string inside
the action list is scanned for ${kind} / ${kind:arg} placeholders
before dispatch. The following kinds are supported:
${env:NAME}— value of theNAMEenvironment variable (empty when unset)${date:FMT}—datetime.now().strftime(FMT); bare${date}yields ISO${uuid}— a freshuuid.uuid4().hex${cwd}—os.getcwd()
Unknown kinds raise SubstitutionException so typos surface loudly
rather than leaking literal ${...} into paths.
- exception automation_file.core.substitution.SubstitutionException[source]
Bases:
FileAutomationExceptionRaised when a
${...}reference names an unknown kind.
- automation_file.core.substitution.substitute(payload)[source]
Return a deep copy of
payloadwith every${...}expanded.
Entry-point plugin discovery.
Third-party packages can register additional actions with
automation_file without the library having to import them directly.
A plugin advertises itself in its pyproject.toml:
[project.entry-points."automation_file.actions"]
my_plugin = "my_plugin:register"
where register is a zero-argument callable returning a
Mapping[str, Callable] — the same shape you would pass to
automation_file.add_command_to_executor().
load_entry_point_plugins() is invoked by
automation_file.core.action_registry.build_default_registry() so
installed plugins populate every freshly-built registry automatically.
Plugin failures are logged and swallowed — one broken plugin must not
break the library for everyone else.
- automation_file.core.plugins.load_entry_point_plugins(register)[source]
Discover and register every
automation_file.actionsentry point.registerreceives one{name: callable}mapping per plugin and is responsible for storing it (typicallyActionRegistry.register_many()). Returns the number of plugins that registered successfully.
Secret provider abstraction.
A secret is a (name -> value) lookup. Three built-in providers compose via
ChainedSecretProvider:
EnvSecretProviderresolves fromos.environ
FileSecretProviderresolves from a directory of per-secret files (Docker / K8s secrets layout)
ChainedSecretProvidertries a list in order
The purpose is to keep secrets out of the config file itself. Callers
write references like ${env:SLACK_WEBHOOK_URL} inside
automation_file.toml; resolve_secret_refs() walks the document
and substitutes. Missing secrets raise SecretNotFoundException
so a typo in a reference never silently becomes an empty string.
- class automation_file.core.secrets.ChainedSecretProvider(providers)[source]
Bases:
SecretProviderTry each child provider in order; return the first non-
Nonevalue.Providers are grouped by
scheme—${env:X}only consultsEnvSecretProviderchildren,${file:X}only consultsFileSecretProviderchildren. Unknown schemes raiseSecretNotFoundExceptionat resolution time.- Parameters:
providers (list[SecretProvider])
- class automation_file.core.secrets.EnvSecretProvider[source]
Bases:
SecretProviderRead secrets from process environment variables.
- class automation_file.core.secrets.FileSecretProvider(root)[source]
Bases:
SecretProviderRead secrets from
<root>/<name>(e.g.,/run/secrets/<name>).- Parameters:
root (str | os.PathLike[str])
- exception automation_file.core.secrets.SecretException[source]
Bases:
FileAutomationExceptionBase for secret-provider failures.
- exception automation_file.core.secrets.SecretNotFoundException[source]
Bases:
SecretExceptionRaised when a
${provider:name}reference cannot be resolved.
- class automation_file.core.secrets.SecretProvider[source]
Bases:
ABCContract for a read-only secret lookup.
- automation_file.core.secrets.default_provider(file_root=None)[source]
Return an env-first chain, optionally augmented with a file provider.
- Parameters:
- Return type:
- automation_file.core.secrets.resolve_secret_refs(value, provider)[source]
Walk
value(dict/list/str) and substitute every${scheme:name}.Strings without references are returned unchanged; non-string scalars pass through untouched. Any unresolved reference raises
SecretNotFoundException— callers should treat this as a hard configuration error rather than continuing with a hole in the config.- Parameters:
value (Any)
provider (ChainedSecretProvider)
- Return type:
Exception hierarchy for automation_file.
All custom exceptions inherit from FileAutomationException so callers can
filter with a single except and still distinguish specific failures.
- exception automation_file.exceptions.AddCommandException[source]
Bases:
FileAutomationExceptionRaised when a command registered into the executor is not callable.
- exception automation_file.exceptions.ArchiveException[source]
Bases:
FileAutomationExceptionRaised when an archive format is unsupported or extraction fails.
- exception automation_file.exceptions.ArgparseException[source]
Bases:
FileAutomationExceptionRaised when the CLI receives no actionable argument.
- exception automation_file.exceptions.BoxException[source]
Bases:
FileAutomationExceptionRaised by the Box backend.
- exception automation_file.exceptions.CASException[source]
Bases:
FileAutomationExceptionRaised by the content-addressable store on integrity / I/O failures.
- exception automation_file.exceptions.CallbackExecutorException[source]
Bases:
FileAutomationExceptionRaised by
CallbackExecutorfor registration / dispatch failures.
- exception automation_file.exceptions.CircuitOpenException[source]
Bases:
FileAutomationExceptionRaised when a circuit breaker is open and short-circuits the protected call.
- exception automation_file.exceptions.DagException[source]
Bases:
FileAutomationExceptionRaised when a DAG action list has a cycle, unknown dep, or duplicate id.
- exception automation_file.exceptions.DataOpsException[source]
Bases:
FileAutomationExceptionRaised by CSV / JSONL / YAML / Parquet helpers.
- exception automation_file.exceptions.DiffException[source]
Bases:
FileAutomationExceptionRaised when diff computation or patch application fails.
- exception automation_file.exceptions.DirNotExistsException[source]
Bases:
FileAutomationExceptionRaised when a required directory is missing.
- exception automation_file.exceptions.ExecuteActionException[source]
Bases:
FileAutomationExceptionRaised by
ActionExecutorwhen an action list cannot be run.
- exception automation_file.exceptions.FileAutomationException[source]
Bases:
ExceptionRoot of the automation_file exception tree.
- exception automation_file.exceptions.FileNotExistsException[source]
Bases:
FileAutomationExceptionRaised when a required source file is missing.
- exception automation_file.exceptions.FsspecException[source]
Bases:
FileAutomationExceptionRaised by the fsspec bridge on missing dependency or backend failures.
- exception automation_file.exceptions.JsonActionException[source]
Bases:
FileAutomationExceptionRaised when JSON action files cannot be read or written.
- exception automation_file.exceptions.LockTimeoutException[source]
Bases:
FileAutomationExceptionRaised when a lock acquire waits past its timeout.
- exception automation_file.exceptions.MCPServerException[source]
Bases:
FileAutomationExceptionRaised by the MCP server bridge when a tool invocation fails.
- exception automation_file.exceptions.OneDriveException[source]
Bases:
FileAutomationExceptionRaised by the OneDrive (Microsoft Graph) backend.
- exception automation_file.exceptions.PathTraversalException[source]
Bases:
FileAutomationExceptionRaised when a user-supplied path escapes the allowed root.
- exception automation_file.exceptions.QueueException[source]
Bases:
FileAutomationExceptionRaised by the persistent action queue on storage / dispatch errors.
- exception automation_file.exceptions.QuotaExceededException[source]
Bases:
FileAutomationExceptionRaised when an action exceeds a configured size or duration quota.
- exception automation_file.exceptions.RateLimitExceededException[source]
Bases:
FileAutomationExceptionRaised when a rate-limited call cannot acquire a token in the allotted wait.
- exception automation_file.exceptions.RetryExhaustedException[source]
Bases:
FileAutomationExceptionRaised when a
@retry_on_transientwrapped call runs out of attempts.
- exception automation_file.exceptions.SMBException[source]
Bases:
FileAutomationExceptionRaised by the SMB/CIFS client on connection / protocol failures.
- exception automation_file.exceptions.TCPAuthException[source]
Bases:
FileAutomationExceptionRaised when a TCP client fails shared-secret authentication.
- exception automation_file.exceptions.TemplateException[source]
Bases:
FileAutomationExceptionRaised when template rendering fails (missing engine, syntax, I/O).
- exception automation_file.exceptions.TextOpsException[source]
Bases:
FileAutomationExceptionRaised by text / binary file helpers (split, merge, sed, encoding_convert).
- exception automation_file.exceptions.TracingException[source]
Bases:
FileAutomationExceptionRaised when OpenTelemetry tracing setup cannot be completed.
- exception automation_file.exceptions.UrlValidationException[source]
Bases:
FileAutomationExceptionRaised when a URL fails scheme / host validation (SSRF guard).
- exception automation_file.exceptions.ValidationException[source]
Bases:
FileAutomationExceptionRaised when an action list fails pre-execution validation.
- exception automation_file.exceptions.VersioningException[source]
Bases:
FileAutomationExceptionRaised by the versioning helpers on retention / I/O failures.
- exception automation_file.exceptions.WebDAVException[source]
Bases:
FileAutomationExceptionRaised by the WebDAV client on transport / protocol failures.
- exception automation_file.exceptions.ZipInputException[source]
Bases:
FileAutomationExceptionRaised when a zip helper receives an unsupported input type.
Module-level logger for automation_file.
A single file_automation_logger is exposed. It writes to
FileAutomation.log in append mode and mirrors every record to stderr via a
custom handler. The handler list is rebuilt only once, even if the module is
reloaded, so tests can import this safely.