Koog Docs with Knit (#542)

Co-authored-by: Inna Teteniuk <[email protected]>
Co-authored-by: Marko Marinkovic <[email protected]>
Co-authored-by: Maria Tigina <[email protected]>

Author: 3 people

.github/workflows/checks.yml

@@ -62,6 +62,7 @@ jobs:
         run: |
           cd ./docs
           pip install mkdocs-material
+          pip install mkdocs-redirects
           mkdocs build
 
   tests:
@@ -151,3 +152,28 @@ jobs:
         env:
           QODANA_TOKEN: ${{ secrets.QODANA_TOKEN_1399872884 }}
           QODANA_ENDPOINT: 'https://qodana.cloud'
+  knit:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-java@v4
+        with:
+          distribution: 'temurin'
+          java-version: 17
+      - uses: gradle/actions/setup-gradle@v4
+      - name: Generate and verify code examples
+        run: |
+          echo "Starting knit generation..."
+          FILES_BEFORE_KNIT=$(find docs/src/ -name "*.kt" 2>/dev/null | wc -l || echo "0")
+          echo "Files before knit: $FILES_BEFORE_KNIT"
+
+          ./gradlew :docs:knit
+
+          FILES_AFTER_KNIT=$(find docs/src/ -name "*.kt" 2>/dev/null | wc -l || echo "0")
+          echo "Files after knit: $FILES_AFTER_KNIT"
+
+          KNIT_GENERATED_FILES=$((FILES_AFTER_KNIT - FILES_BEFORE_KNIT))
+          echo "Knit generated $KNIT_GENERATED_FILES files"
+
+          echo "Starting assemble..."
+          ./gradlew :docs:assemble

.github/workflows/push-to-main.yml

@@ -28,4 +28,5 @@ jobs:
         run: |
           cd ./docs
           pip install mkdocs-material
-          mkdocs gh-deploy
+          pip install mkdocs-redirects
+          mkdocs gh-deploy

.gitignore

@@ -7,3 +7,4 @@ env.properties
 .qodana/code-coverage
 local.properties
 /kotlin-js-store/yarn.lock
+docs/src/main/kotlin/*.kt

docs/README.md

@@ -0,0 +1,102 @@
+# Koog Documentation
+
+This module contains documentation for the Koog framework, including user guides, API references, prompting guidelines, and other static files.
+
+## Module Structure
+
+The docs module is organized as follows:
+
+- **docs/** - Contains markdown files with user documentation
+- **overrides/** - Custom overrides for the MkDocs theme
+- **prompt/** - Prompting guidelines with extensions for popular modules
+- **src/** - Knit generated source code from documentation code snippets, should not be commited
+
+## Documentation System
+
+### MkDocs
+
+The documentation is built using [MkDocs](https://www.mkdocs.org/) with the Material theme. The configuration is defined in `mkdocs.yml`, which specifies:
+
+- Navigation structure
+- Theme configuration
+- Markdown extensions
+- Repository links
+
+The documentation is available at [https://docs.koog.ai/](https://docs.koog.ai/).
+
+### Docs Code Snippets Verification
+
+To ensure code snippets in documentation are compilable and up-to-date with the latest framework version, the [kotlinx-knit](https://github.com/Kotlin/kotlinx-knit) library is used.
+
+Knit provides a Gradle plugin that extracts specially annotated Kotlin code snippets from markdown files and generates Kotlin source files.
+
+#### How to fix docs?
+1. Run knit to extract code snippets to /src/main/kotlin:
+```
+./gradlew :docs:knit
+```
+2. Run assemble to get compilation arrows:
+```
+./gradlew :docs:assemble
+```
+3. Navigate to the file with the compilation error `example-[md-file-name]-[index].kt`
+4. Fix the error in this file
+5. Navigate to the code snippet in Markdown `md-file-name.md` by searing `<!--- KNIT example-[md-file-name]-[index].kt` -->`
+6. Update the code snippet to reflect the changes in kt file
+   * Update dependencies (usually they are provided in `<!--- INCLUDE -->` section)
+   * Edit code (don't forget about tabulation when you just copy paste from kt)
+
+#### How to annotate docs?
+
+To annotate new Kotlin code snippets in Markdown and make them compilable:
+1. Put an example annotation comment (`<!--- KNIT example-[md-file-name]-01.kt -->`) after every code block. 
+It's not obligated to put right inexes, just set the `01` for each example, and they will be updated automatically after first knit run
+```
+    ```kotlin
+    val agent = AIAgent(...)
+    ```
+    <!--- KNIT example-[md-file-name]-01.kt -->
+```
+2. In case you need some imports, add include comment (`<!--- INCLIDE ... -->`)
+
+```
+    <!--- INCLUDE
+    import ai.koog.agents.core.agent.AIAgent
+    -->
+    ```kotlin
+    val agent = AIAgent(...)
+    ```
+    <!--- KNIT example-[md-file-name]-01.kt -->
+```
+3. In case you need to whap your code into `main` or other function imports, 
+use include comment (`<!--- INCLIDE ... -->`) for prefix and suffix comment (`<!--- SUFFIX ... -->`) for suffix
+```
+    <!--- INCLUDE
+    import ai.koog.agents.core.agent.AIAgent
+    fun main() {
+    -->
+    <!--- SUFFIX
+    }
+    -->
+    ```kotlin
+    val agent = AIAgent(...)
+    ```
+    <!--- KNIT example-[md-file-name]-01.kt -->
+```
+
+For more information, follow the examples in the [kotlinx-knit](https://github.com/Kotlin/kotlinx-knit) repository 
+or refer to already annotated code snippets in the documentation.
+
+### API Documentation
+
+API reference documentation is generated using [Dokka](https://github.com/Kotlin/dokka), a documentation engine for Kotlin. The API documentation is built with:
+
+```
+./gradlew dokkaGenerate
+```
+
+The generated API documentation is deployed to [https://api.koog.ai/](https://api.koog.ai/).
+
+## Prompts
+
+In the [prompt](./prompt) directory, prompting guidelines with extensions for popular modules are stored. These guidelines help users create effective prompts for different LLM models and use cases.

docs/build.gradle.kts

@@ -0,0 +1,24 @@
+group = rootProject.group
+version = rootProject.version
+
+plugins {
+    id("ai.kotlin.jvm")
+    alias(libs.plugins.kotlin.serialization)
+    alias(libs.plugins.knit)
+}
+
+dependencies {
+    implementation(project(":agents:agents-test"))
+    implementation(project(":koog-agents"))
+    implementation(libs.opentelemetry.exporter.otlp)
+    implementation(libs.opentelemetry.exporter.logging)
+}
+
+knit {
+    rootDir = project.rootDir
+    files = fileTree("docs/") {
+        include("**/*.md")
+    }
+    moduleDocs = "docs/modules.md"
+    siteRoot = "https://docs.koog.ai/"
+}

docs/docs/agent-events.md

@@ -38,43 +38,65 @@ ai/agents/agents-features/agents-features-event-handler/ai.koog.agents.local.fea
 
 To install the feature and configure event handlers for the agent, do the following:
 
-```kotlin
-{
-    install(EventHandler){
-        // Define event handlers here
-        onToolCall = { stage, tool, toolArgs ->
-            // Handle tool call event
-        }
+<!--- INCLUDE
+import ai.koog.agents.core.agent.AIAgent
+import ai.koog.agents.features.eventHandler.feature.handleEvents
+import ai.koog.prompt.executor.llms.all.simpleOllamaAIExecutor
+import ai.koog.prompt.llm.OllamaModels
 
-        onAgentFinished = { strategyName, result ->
-            // Handle event triggered when the agent completes its execution
-        }
+val agent = AIAgent(
+    executor = simpleOllamaAIExecutor(),
+    llmModel = OllamaModels.Meta.LLAMA_3_2,
+) {
+-->
+<!--- SUFFIX 
+} 
+-->
 
-        // Define other event handlers
+```kotlin
+handleEvents {
+    // Handle tool calls
+    onToolCall { eventContext ->
+        println("Tool called: ${eventContext.tool} with args ${eventContext.toolArgs}")
+    }
+    // Handle event triggered when the agent completes its execution
+    onAgentFinished { eventContext ->
+        println("Agent finished with result: ${eventContext.result}")
     }
+
+    // Other event handlers
 }
 ```
+<!--- KNIT example-events-01.kt -->
 
 For more details about event handler configuration, see [API reference](https://api.koog.ai/agents/agents-features/agents-features-event-handler/ai.koog.agents.local.features.eventHandler.feature/-event-handler-config/index.html).
 
 You can also set up event handlers using the `handleEvents` extension function when creating an agent.
 This function also installs the event handler feature and configures event handlers for the agent. Here is an example:
 
+<!--- INCLUDE
+import ai.koog.agents.core.agent.AIAgent
+import ai.koog.agents.features.eventHandler.feature.handleEvents
+import ai.koog.prompt.executor.llms.all.simpleOllamaAIExecutor
+import ai.koog.prompt.llm.OllamaModels
+-->
 ```kotlin
 val agent = AIAgent(
-    // Initialization options
+    executor = simpleOllamaAIExecutor(),
+    llmModel = OllamaModels.Meta.LLAMA_3_2,
 ){
     handleEvents {
         // Handle tool calls
-        onToolCall = { stage, tool, toolArgs ->
-            println("Tool called: ${tool.name} with args $toolArgs")
+        onToolCall { eventContext ->
+            println("Tool called: ${eventContext.tool} with args ${eventContext.toolArgs}")
         }
         // Handle event triggered when the agent completes its execution
-        onAgentFinished = { strategyName, result ->
-            println("Agent finished with result: $result")
+        onAgentFinished { eventContext ->
+            println("Agent finished with result: ${eventContext.result}")
         }
 
         // Other event handlers
     }
 }
 ```
+<!--- KNIT example-events-02.kt -->