Project Documentation

🚀 1. Overview

The UnityShaderParser project is a comprehensive library designed for parsing Unity shaders, encompassing various distinct components. At its core, the library consists of a parser for ShaderLab, Unity's proprietary Shader Domain-Specific Language (DSL), as well as a parser for High-Level Shading Language (HLSL), which is embedded within ShaderLab. Additionally, it features a preprocessor for handling macros prior to parsing and a framework for analyzing syntax trees using the visitor pattern, allowing for edits to the corresponding source code.

The primary purpose of the UnityShaderParser project is to provide a robust and reliable solution for parsing and analyzing Unity shaders, enabling developers to create a wide range of applications, from shader code interpretation and execution to syntax analysis and modification. The library has been utilized in various projects, including the HLSLInterpreter, a library for running shader code on the CPU, and the Slang shading language plugin for Unity. Its versatility and effectiveness have also been demonstrated in numerous examples and tests, showcasing its potential for a broad range of use cases.

From a technical standpoint, the UnityShaderParser project is built using a .NET-based stack, with the core library implemented in C#. The project's continuous integration and continuous deployment (CI/CD) pipeline is managed through GitHub Actions, as evidenced by the .github/workflows/test-ci.yml file. This pipeline automates the build, testing, and deployment process, ensuring that the library is consistently reliable and up-to-date. The use of .NET 6.0 and the dotnet CLI facilitates efficient development, testing, and deployment of the library.

Tech Stack at a Glance

In terms of state management, styling, and database libraries, the UnityShaderParser project does not appear to have specific dependencies or integrations. Its primary focus is on parsing and analyzing Unity shaders, and it achieves this through its own custom implementations and frameworks. The library's design and architecture prioritize simplicity, reliability, and performance, making it an attractive solution for developers seeking a robust and efficient shader parsing and analysis tool. Overall, the UnityShaderParser project represents a valuable resource for the development community, offering a powerful and flexible solution for working with Unity shaders.

📂 2. Project Structure

Directory Structure

The codebase contains the following primary files and directories:

• .editorconfig
• .github/workflows/test-ci.yml
• .gitignore
• LICENSE
• README.md
• UnityShaderParser.Experiments/Program.cs
• UnityShaderParser.Experiments/UnityShaderParser.Experiments.csproj
• UnityShaderParser.Tests/HLSLParserTests.cs
• UnityShaderParser.Tests/PreProcessorTests.cs
• UnityShaderParser.Tests/ShaderLabParserTests.cs
• UnityShaderParser.Tests/ShaderParserTests.cs
• UnityShaderParser.Tests/TestShaders/FxDis/readme.txt
• UnityShaderParser.Tests/TestShaders/FxDis/test.hlsl
• UnityShaderParser.Tests/TestShaders/HlslCrossCompiler/ds5/basic.hlsl
• UnityShaderParser.Tests/TestShaders/HlslCrossCompiler/ds5/basic.knowngood
• UnityShaderParser.Tests/TestShaders/HlslCrossCompiler/hs5/basic.hlsl
• UnityShaderParser.Tests/TestShaders/HlslCrossCompiler/ps4/fxaa.hlsl
• UnityShaderParser.Tests/TestShaders/HlslCrossCompiler/ps4/primID.hlsl
• UnityShaderParser.Tests/TestShaders/HlslCrossCompiler/ps5/conservative_depth.hlsl
• UnityShaderParser.Tests/TestShaders/HlslCrossCompiler/ps5/interface_arrays.hlsl

Generated via local analysis.

🏗️ 3. Architecture & Data Flow

Deep Architecture Analysis of UnityShaderParser

1. Architecture Reasoning & Module Interactions

Core Architecture Overview

UnityShaderParser follows a modular compiler architecture with distinct parsing layers for different shader languages (ShaderLab and HLSL) while sharing common infrastructure. The system is designed as a recursive descent parser with strong separation of concerns between lexical analysis, parsing, and syntax tree manipulation.

Key Design Patterns & Principles

1. Lexer/Parser Separation Pattern:

   • Each language (ShaderLab, HLSL) has dedicated lexers (ShaderLabLexer.cs, HLSLLexer.cs) that tokenize input
   • Corresponding parsers (ShaderLabParser.cs, HLSLParser.cs) build syntax trees from tokens
   • Follows the Compiler Compiler pattern where lexers produce tokens consumed by parsers

2. Visitor Pattern:

   • HLSLSyntaxVisitor.cs and ShaderLabSyntaxVisitor.cs provide interfaces for traversing syntax trees
   • Enables analysis, transformation, and code generation without modifying core syntax classes
   • Used extensively in tests for validation and in real-world tools for shader analysis

3. Composite Syntax Elements:

   • BaseSyntaxElements.cs defines the core syntax tree structure with SyntaxElement base class
   • Language-specific syntax elements inherit from base classes (HLSLSyntaxElements.cs, ShaderLabSyntaxElements.cs)
   • Implements the Composite Pattern allowing tree structures of arbitrary depth

4. Preprocessor Pipeline:

   • HLSLPreProcessor.cs handles macro expansion and conditional compilation
   • Uses IPreProcessorIncludeResolver for file inclusion resolution
   • Implements Chain of Responsibility for handling different preprocessor directives

5. Editor Integration:

   • BaseLexer.cs and BaseParser.cs provide common infrastructure for both languages
   • ShaderParser.cs acts as a Facade providing unified entry point for parsing either language

Module Interactions & Data Flow

1. Parsing Pipeline:

   Source File → Lexer → Token Stream → Parser → Syntax Tree
   

2. Preprocessing Pipeline:

   HLSL Source → PreProcessor → Expanded Source → Lexer → Parser
   

3. Analysis Pipeline:

   Syntax Tree → Visitor → Analysis Results
   

4. Transformation Pipeline:

   Syntax Tree → Visitor → Modified Syntax Tree → Printer → Output Source
   

Advanced Design Considerations

1. Error Recovery:

   • Both lexers and parsers implement panic mode recovery to continue parsing after errors
   • BaseLexer.cs tracks error positions for precise error reporting

2. Memory Efficiency:

   • Syntax elements store source spans rather than full text
   • Uses flyweight pattern for common syntax elements

3. Extensibility:

   • Virtual methods in base classes allow language-specific behavior
   • Visitor pattern enables adding new operations without modifying syntax classes

4. Cross-Language Support:

   • Common infrastructure handles both ShaderLab and HLSL
   • Type system bridges HLSL and ShaderLab concepts where needed

2. Data Flow & State Transitions

3. Mermaid.js Architecture Diagram

Key Component Relationships

1. Core Infrastructure:

   • BaseLexer.cs/BaseParser.cs: Foundation for all language parsers
   • BaseSyntaxElements.cs: Defines the syntax tree structure
   • ShaderParser.cs: Unified entry point for parsing either language

2. Language-Specific Components:

   • ShaderLab: Complete parser stack from lexing to printing
   • HLSL: Complete parser stack with additional preprocessing
   • Both share the same core infrastructure but have language-specific syntax

3. Preprocessing:

   • Only applies to HLSL (ShaderLab has minimal preprocessing)
   • Handles macro expansion and conditional compilation
   • Includes file resolution through IPreProcessorIncludeResolver

4. Visitor Pattern:

   • Language-specific visitors (ShaderLabSyntaxVisitor.cs, HLSLSyntaxVisitor.cs)
   • Enable analysis and transformation of syntax trees
   • Used extensively in tests and real-world tools

5. Printers:

   • ShaderLabPrinter.cs and HLSLPrinter.cs convert AST back to source
   • Enable round-trip parsing and code generation

6. Test Infrastructure:

   • Comprehensive test suite covering both languages
   • Test shaders from multiple sources (Nvidia, Unity, DirectX SDK)
   • Validates parsing, preprocessing, and printing

This architecture demonstrates a well-structured compiler system with clear separation of concerns, enabling robust parsing of Unity shaders while maintaining extensibility for future enhancements.

📈 System Architecture Diagram

🛠️ 4. Function Breakdowns

.github/workflows/test-ci.yml

README.md

However, the library itself is not provided, so a detailed function-level code breakdown cannot be performed. The actual code of the UnityShaderParser library would need to be analyzed to provide a comprehensive breakdown of its functions, parameters, logic, and impact.

Here is an example of what the breakdown might look like for a hypothetical ShaderParser class: