Skip to main content

Orchestration & Separate Runners

The engine provides separate, modular runners for different workflow types (workflow, feature, solution), enabling orchestration with shared context. All runners load and execute WSL files from disk.

Architecture​

Runner Types​

The engine includes three built-in runner types, each designed for a specific level of orchestration:

1. WorkflowRunner​

  • Handles execution of workflow type flows
  • Loads workflow_name.wsl files from disk
  • Executes individual workflow steps
  • Can run standalone or with shared context

2. FeatureRunner​

  • Handles execution of feature type flows
  • Loads feature_name.wsl files from disk
  • Orchestrates multiple workflows in a chain
  • All workflows in a feature share the same WorkerSessionContext
  • Provides workflow chaining capabilities

3. SolutionRunner​

  • Handles execution of solution type flows
  • Loads solution_name.wsl files from disk
  • Orchestrates both features and workflows
  • All features and workflows in a solution share the same WorkerSessionContext
  • Provides feature and workflow chaining capabilities

Shared Context​

All orchestrated components share the same WorkerSessionContext, enabling seamless data flow between workflows, features, and solutions.

// Values set in one workflow
workflowContext["shared_value"] = "data"

// Are accessible in subsequent workflows
value := workflowContext["shared_value"] // "data"

Basic Orchestration​

Feature Orchestrating Workflows​

A feature can orchestrate multiple workflows that run sequentially:

// feature_data_processing.wsl
module features

feature data_processing {
    start: Step1
    
    state Step1 {
        action workflow process_step1
        on success -> Step2
    }
    
    state Step2 {
        action workflow process_step2
        on success -> Step3
    }
    
    state Step3 {
        action workflow finalize
        on success -> Done
    }
    
    state Done {
        end ok
    }
}

Solution Orchestrating Features and Workflows​

A solution can orchestrate both features and workflows:

// solution_app.wsl
module solutions

solution complete_app {
    start: RunFeature
    
    state RunFeature {
        action feature data_processing
        on success -> RunWorkflow
    }
    
    state RunWorkflow {
        action workflow reporting
        on success -> Complete
    }
    
    state Complete {
        action response/ResponseValue(message: "Solution complete", statusCode: 200)
        end ok
    }
}

Advanced Orchestration Patterns​

Sequential Workflow Chain​

Execute workflows one after another, sharing context:

feature user_onboarding {
    start: CreateAccount
    
    state CreateAccount {
        action workflow create_user
        on success -> SendWelcome
    }
    
    state SendWelcome {
        action workflow send_welcome_email
        on success -> SetupProfile
    }
    
    state SetupProfile {
        action workflow initialize_profile
        on success -> Complete
    }
    
    state Complete {
        end ok
    }
}

Mixed Orchestration​

Combine features and workflows in a solution:

solution ecommerce_platform {
    start: SetupUsers
    
    state SetupUsers {
        action feature user_management
        on success -> SetupCatalog
    }
    
    state SetupCatalog {
        action workflow product_catalog_init
        on success -> SetupPayments
    }
    
    state SetupPayments {
        action feature payment_processing
        on success -> StartCheckout
    }
    
    state StartCheckout {
        action workflow checkout_flow
        on success -> SystemReady
    }
    
    state SystemReady {
        end ok
    }
}

Conditional Orchestration​

Use conditional transitions with orchestration:

feature data_pipeline {
    start: ValidateInput
    
    state ValidateInput {
        action workflow input_validation
        on success -> CheckDataSize
    }
    
    state CheckDataSize {
        action utils/check.DataSize() as SizeCheck
        on success -> RouteBySize(SizeCheck)
    }
    
    state RouteBySize(SizeCheck) {
        // If large dataset, use parallel processing
        action workflow parallel_processor
        on success -> Complete
        on error -> UseSingleProcessor
    }
    
    state UseSingleProcessor {
        action workflow single_processor
        on success -> Complete
    }
    
    state Complete {
        end ok
    }
}

Context Sharing Example​

Here's how data flows through orchestrated workflows:

// workflow1.wsl - Sets shared data
workflow extract_data {
    start: Fetch
    
    state Fetch {
        action data/fetch.FromAPI() as ApiData
        on success -> StoreInContext(ApiData)
    }
    
    state StoreInContext(ApiData) {
        action context/set.Value(key: "extracted_data", value: $ApiData)
        end ok
    }
}

// workflow2.wsl - Uses shared data
workflow transform_data {
    start: LoadFromContext
    
    state LoadFromContext {
        action context/get.Value(key: "extracted_data") as ExtractedData
        on success -> Transform(ExtractedData)
    }
    
    state Transform(ExtractedData) {
        action transform/apply.Transformation(data: $ExtractedData)
        on success -> SaveResult
    }
    
    state SaveResult {
        end ok
    }
}

// feature.wsl - Orchestrates both workflows
feature etl_pipeline {
    start: Extract
    
    state Extract {
        action workflow extract_data
        on success -> Transform
    }
    
    state Transform {
        action workflow transform_data
        on success -> Complete
    }
    
    state Complete {
        end ok
    }
}

Runner Factory​

The RunnerFactory automatically selects the appropriate runner based on flow type:

factory := workflow.NewRunnerFactory(config, app)
runner, err := factory.CreateRunner(flowType) // "workflow", "feature", or "solution"
if err != nil {
    log.Fatal(err)
}

result := runner.Run(ctx, options)

Custom Runners​

Register custom runners for new workflow types:

// Register a custom runner
workflow.RegisterCustomRunner("pipeline", func(config domain.WorkflowConfigItem, app pkg.Application) workflow.Runner {
    return &PipelineRunner{
        config: config,
        app:    app,
    }
})

// Use the custom runner
factory := workflow.NewRunnerFactory(config, app)
runner, err := factory.CreateRunner("pipeline")

Custom Runner Implementation​

type PipelineRunner struct {
    config domain.WorkflowConfigItem
    app    pkg.Application
}

func (r *PipelineRunner) Run(ctx context.Context, options *domain.Options) map[string]*workflow.WorkerResponse {
    // Load WSL file
    wslContent, err := r.loadWSLFile(options.WorkflowName)
    if err != nil {
        return errorResponse(err)
    }
    
    // Parse and execute
    result := r.executePipeline(ctx, wslContent, options)
    return result
}

func (r *PipelineRunner) loadWSLFile(name string) (string, error) {
    path := filepath.Join("runtime/workflows", name, name+".wsl")
    content, err := os.ReadFile(path)
    return string(content), err
}

Complete Example: E-commerce System​

Workflow Layer​

// workflow_user_auth.wsl
workflow user_authentication {
    start: ValidateCredentials
    
    state ValidateCredentials {
        action auth/validate.Check()
        on success -> GenerateToken
        on error -> AuthFailed
    }
    
    state GenerateToken {
        action auth/token.Create()
        end ok
    }
    
    state AuthFailed {
        end error
    }
}

Feature Layer​

// feature_user_mgmt.wsl
feature user_management {
    start: Authenticate
    
    state Authenticate {
        action workflow user_authentication
        on success -> LoadProfile
        on error -> HandleAuthError
    }
    
    state LoadProfile {
        action workflow user_profile_load
        on success -> Ready
    }
    
    state Ready {
        end ok
    }
    
    state HandleAuthError {
        end error
    }
}

Solution Layer​

// solution_ecommerce.wsl
solution ecommerce_platform {
    start: InitializeUsers
    
    state InitializeUsers {
        action feature user_management
        on success -> InitializeProducts
    }
    
    state InitializeProducts {
        action feature product_catalog
        on success -> InitializePayments
    }
    
    state InitializePayments {
        action feature payment_processing
        on success -> SystemReady
    }
    
    state SystemReady {
        action response/ResponseValue(message: "Platform ready", statusCode: 200)
        end ok
    }
}

Benefits​

1. Separation of Concerns​

Each runner handles its specific type independently, making code more maintainable.

2. Modularity​

Runners can be composed, reused, and extended for different use cases.

3. Shared Context​

Seamless data flow between orchestrated components without explicit parameter passing.

4. WSL File Execution​

Clean separation between workflow definition (WSL files) and execution (runners).

5. Extensibility​

New workflow types can be added via the Runner Registry without modifying core code.

6. Type Safety​

Proper validation ensures each workflow type is handled correctly.

Best Practices​

1. Use Appropriate Levels​

  • Workflows - For single-purpose, focused tasks
  • Features - For related workflows that work together
  • Solutions - For complete systems combining multiple features

2. Keep Workflows Small​

Each workflow should do one thing well:

// Good - focused workflow
workflow validate_email {
    start: CheckFormat
    state CheckFormat { ... }
    state VerifyDomain { ... }
}

// Avoid - too much in one workflow
workflow handle_everything {
    start: DoEverything
    // 50+ states...
}

3. Name Files Clearly​

Use descriptive names that match the workflow name:

runtime/workflows/
  user_authentication/
    user_authentication.wsl
  payment_processing/
    payment_processing.wsl

4. Document Context Data​

Document what data each workflow expects and provides in shared context:

// Expects: context["user_id"]
// Provides: context["user_profile"]
workflow load_user_profile {
    start: FetchProfile
    // ...
}

5. Handle Errors Gracefully​

Include error handling at each orchestration level:

feature data_import {
    start: Import
    
    state Import {
        action workflow import_data
        on success -> Validate
        on error -> HandleImportError
    }
    
    state HandleImportError {
        action workflow cleanup_partial_import
        end error
    }
    
    state Validate {
        action workflow validate_imported_data
        on success -> Complete
        on error -> HandleValidationError
    }
    
    state HandleValidationError {
        action workflow rollback_import
        end error
    }
    
    state Complete {
        end ok
    }
}