Reference

Configuration

Complete reference for m1f configuration files and formats

m1f uses YAML configuration files for auto-bundling and presets. This reference covers all configuration options and file formats.

Configuration Files Overview

m1f uses two main types of configuration files:

  1. Auto-bundle Configuration (.m1f.config.yml) - Defines automated bundle creation
  2. Preset Configuration (*.m1f-presets.yml) - Defines file-specific processing rules

Auto-Bundle Configuration

File Location

m1f searches for .m1f.config.yml starting from the current directory and moving up the directory tree to the root.

Basic Structure

# Global settings that apply to all bundles
global:
  global_excludes:
    - "**/*.pyc"
    - "**/*.log"
    - "**/tmp/**"

# Bundle definitions
bundles:
  docs:
    description: "Project documentation"
    output: "m1f/docs/manual.txt"
    sources:
      - path: "docs"
        include_extensions: [".md", ".txt"]

  code:
    description: "Source code bundle"
    output: "m1f/src/code.txt"
    sources:
      - path: "src"
        include_extensions: [".py", ".js", ".ts"]

Global Settings

global:
  # Default exclusions applied to all bundles
  global_excludes:
    - "**/*.pyc"
    - "**/*.log"
    - "**/node_modules/**"
    - "**/.git/**"
    
  # Default settings for all bundles
  default_separator_style: "Detailed"
  default_max_file_size: "1MB"
  default_security_check: "warn"
  default_encoding: "utf-8"

Bundle Configuration

Each bundle in the bundles section can have the following properties:

Basic Properties

bundles:
  my-bundle:
    description: "Bundle description"        # Required
    output: "path/to/output.txt"            # Required
    group: "optional-group"                 # Optional grouping
    enabled: true                           # Optional (default: true)
    enabled_if_exists: "file.txt"           # Optional conditional enabling

Source Configuration

sources:
  - path: "src"                             # Required
    include_extensions: [".py", ".js"]      # Optional
    exclude_extensions: [".pyc", ".pyo"]    # Optional
    includes: ["pattern/**"]                # Optional gitignore-style patterns
    excludes: ["test_*.py", "*.spec.js"]    # Optional gitignore-style patterns
    max_file_size: "500KB"                  # Optional

Processing Options

# Output formatting
separator_style: "Markdown"                 # Optional
line_ending: "lf"                          # Optional

# Security
security_check: "abort"                    # Optional: abort, skip, warn

# File handling
include_dot_paths: false                   # Optional
include_binary_files: false               # Optional
include_symlinks: false                    # Optional
remove_scraped_metadata: true             # Optional

# Archive creation
create_archive: true                       # Optional
archive_type: "zip"                        # Optional: zip, tar.gz

# Output control
force: true                                # Optional
minimal_output: false                      # Optional
add_timestamp: false                       # Optional
filename_mtime_hash: false                 # Optional

Preset Integration

# Apply presets to bundles
preset: "presets/my-presets.yml"           # Optional
preset_group: "production"                 # Optional

Complete Bundle Example

bundles:
  production-bundle:
    description: "Production-ready code bundle"
    group: "production"
    output: "dist/production.txt"
    enabled_if_exists: "package.json"
    
    # Source configuration
    sources:
      - path: "src"
        include_extensions: [".js", ".ts", ".jsx", ".tsx"]
        excludes: 
          - "**/*.test.*"
          - "**/*.spec.*"
          - "**/test/**"
        max_file_size: "1MB"
      
      - path: "docs"
        include_extensions: [".md"]
        excludes: ["**/draft/**"]
    
    # Processing options
    separator_style: "MachineReadable"
    security_check: "abort"
    create_archive: true
    archive_type: "tar.gz"
    minimal_output: true
    add_timestamp: true
    
    # Preset integration
    preset: "presets/production.m1f-presets.yml"
    preset_group: "minified"

Preset Configuration

File Naming

Preset files should end with .m1f-presets.yml or .m1f-presets.yaml.

Basic Structure

# Preset group
group_name:
  description: "Description of this preset group"
  enabled: true
  priority: 10                              # Higher priority = checked first
  base_path: "src"                          # Optional base path for patterns

  # Global settings for this group
  global_settings:
    # Any CLI option can be specified here
    security_check: "warn"
    max_file_size: "1MB"
    separator_style: "Detailed"

  # File-specific presets
  presets:
    preset_name:
      description: "Description of this preset"
      extensions: [".py", ".js"]            # File extensions to match
      patterns: ["src/**/*.py"]             # Glob patterns to match
      actions: ["strip_comments"]           # Processing actions to apply

Global Settings in Presets

Any CLI option can be specified in the global settings:

global_settings:
  # Input/Output
  separator_style: "Detailed"
  line_ending: "lf"
  
  # File filtering
  include_extensions: [".py", ".js", ".md"]
  exclude_extensions: [".pyc", ".pyo"]
  include_dot_paths: false
  include_binary_files: false
  include_symlinks: false
  max_file_size: "1MB"
  
  # Security
  security_check: "warn"
  
  # Archive
  create_archive: false
  archive_type: "zip"
  
  # Output control
  force: false
  minimal_output: false
  add_timestamp: false
  remove_scraped_metadata: true
  
  # Encoding
  convert_to_charset: "utf-8"
  abort_on_encoding_error: false
  prefer_utf8_for_text_files: true

Extension-Specific Settings

Override settings for specific file extensions:

global_settings:
  extensions:
    .py:
      security_check: "abort"
      max_file_size: "500KB"
      actions: ["strip_comments"]
    
    .js:
      security_check: "warn"
      max_file_size: "100KB"
      actions: ["minify"]
    
    .md:
      security_check: null                  # Disable security check
      actions: ["remove_empty_lines"]
    
    .env:
      security_check: "abort"
      include_binary_files: false

Preset Actions

Available processing actions:

presets:
  my_preset:
    extensions: [".py"]
    actions:
      - "strip_comments"                    # Remove comments
      - "remove_empty_lines"                # Remove empty lines
      - "compress_whitespace"               # Normalize whitespace
      - "minify"                           # Minify content
      - "strip_tags"                       # Remove HTML tags
      - "custom"                           # Apply custom processor

Custom Processors

presets:
  custom_processing:
    extensions: [".log"]
    actions: ["custom"]
    custom_processor: "truncate"
    processor_args:
      max_lines: 1000
      add_marker: true

Available custom processors:

  • truncate: Limit content length
  • redact_secrets: Remove sensitive data patterns
  • extract_functions: Extract function definitions (Python)

Complete Preset Example

web_project:
  description: "Web project processing rules"
  enabled: true
  priority: 10
  
  global_settings:
    security_check: "warn"
    max_file_size: "1MB"
    separator_style: "Markdown"
    
    # Extension-specific settings
    extensions:
      .js:
        max_file_size: "100KB"
        actions: ["minify"]
      .css:
        max_file_size: "50KB"
        actions: ["minify"]
      .html:
        actions: ["minify", "strip_tags"]
      .md:
        security_check: null
        actions: ["remove_empty_lines"]
  
  presets:
    # JavaScript files
    javascript:
      extensions: [".js", ".jsx"]
      patterns: ["src/**/*.js", "!src/**/*.min.js"]
      actions: ["strip_comments", "compress_whitespace"]
      
    # CSS files
    styles:
      extensions: [".css", ".scss", ".sass"]
      actions: ["minify"]
      
    # HTML templates
    templates:
      extensions: [".html", ".htm"]
      actions: ["minify", "strip_tags"]
      strip_tags: ["script", "style"]
      preserve_tags: ["pre", "code"]
      
    # Configuration files
    config:
      extensions: [".json", ".yml", ".yaml"]
      actions: ["compress_whitespace"]
      
    # Large data files
    data:
      extensions: [".csv", ".tsv", ".data"]
      actions: ["custom"]
      custom_processor: "truncate"
      processor_args:
        max_lines: 100
        add_marker: true
        
    # Sensitive files
    secrets:
      patterns: ["**/.env*", "**/config/secrets*"]
      actions: ["custom"]
      custom_processor: "redact_secrets"
      processor_args:
        patterns:
          - '(?i)api[_-]?key\s*[:=]\s*["\']?[\w-]+["\']?'
          - '(?i)password\s*[:=]\s*["\']?[\w-]+["\']?'

Configuration Validation

m1f validates configuration files on load and provides helpful error messages:

Common Validation Errors

  1. Invalid YAML syntax

    Error: Invalid YAML syntax in .m1f.config.yml:
Line 5: mapping values are not allowed here

  2. Missing required fields

    Error: Bundle 'my-bundle' is missing required field 'output'

  3. Invalid file paths

    Error: Bundle 'my-bundle' output path contains invalid characters

  4. Circular dependencies

    Error: Circular dependency detected in preset loading

Environment Variables

Configuration files can use environment variable substitution:

bundles:
  production:
    output: "${OUTPUT_DIR}/production.txt"
    sources:
      - path: "${SOURCE_PATH}"
        max_file_size: "${MAX_FILE_SIZE:-1MB}"

Configuration Inheritance

Preset Loading Order

  1. Global Presets (~/.m1f/global-presets.yml) - Lowest priority
  2. User Presets (~/.m1f/presets/*.yml) - Medium priority
  3. Project Presets (via --preset parameter) - Highest priority

Override Behavior

Later configurations override earlier ones:

# base.yml
web_project:
  global_settings:
    security_check: "warn"
    max_file_size: "1MB"

# project.yml (overrides base.yml)
web_project:
  global_settings:
    security_check: "abort"    # Overrides "warn"
    # max_file_size: "1MB"     # Inherited from base.yml

Configuration Templates

Minimal Configuration

bundles:
  simple:
    description: "Simple bundle"
    output: "bundle.txt"
    sources:
      - path: "."

Multi-Environment Configuration

bundles:
  development:
    description: "Development bundle"
    group: "dev"
    output: "dev/bundle.txt"
    sources:
      - path: "src"
    security_check: "warn"
  
  production:
    description: "Production bundle"
    group: "prod"
    output: "prod/bundle.txt"
    sources:
      - path: "src"
        excludes: ["**/*.test.*"]
    security_check: "abort"
    minimal_output: true

Large Project Configuration

global:
  global_excludes:
    - "**/node_modules/**"
    - "**/.git/**"
    - "**/*.pyc"
    - "**/*.log"

bundles:
  backend:
    description: "Backend API code"
    group: "backend"
    output: "bundles/backend.txt"
    sources:
      - path: "api"
        include_extensions: [".py"]
        max_file_size: "500KB"
    security_check: "abort"
  
  frontend:
    description: "Frontend application"
    group: "frontend"
    output: "bundles/frontend.txt"
    sources:
      - path: "webapp"
        include_extensions: [".js", ".jsx", ".ts", ".tsx"]
        max_file_size: "200KB"
    preset: "presets/frontend.m1f-presets.yml"
  
  documentation:
    description: "Project documentation"
    group: "docs"
    output: "bundles/docs.txt"
    sources:
      - path: "docs"
        include_extensions: [".md", ".rst"]
    separator_style: "Markdown"
    security_check: "skip"

Best Practices

  1. Use descriptive names for bundles and presets
  2. Group related bundles using the group property
  3. Set appropriate security levels for different environments
  4. Use conditional bundles with enabled_if_exists for optional features
  5. Leverage preset inheritance to avoid duplication
  6. Test configurations with small file sets first
  7. Version control your configuration files
  8. Document custom processors and complex patterns
  • Auto Bundle - Using configuration files for automated bundling
  • Presets - Advanced preset configuration
  • CLI Reference - Command-line options that can be used in configuration
  • Security - Security considerations in configuration
Previous
Presets
Next
s1f - File Splitter