8000 GitHub - AlexB-mgtc/ZLinq: Zero allocation LINQ with Span and LINQ to SIMD, LINQ to Tree (FileSystem, Json, GameObject, etc.) for all .NET platforms and Unity.
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
forked from Cysharp/ZLinq

Zero allocation LINQ with Span and LINQ to SIMD, LINQ to Tree (FileSystem, Json, GameObject, etc.) for all .NET platforms and Unity.

License

Notifications You must be signed in to change notification settings

AlexB-mgtc/ZLinq

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ZLinq

Zero allocation LINQ with Span and LINQ to SIMD, LINQ to Tree (FileSystem, Json, GameObject, etc.) for all .NET platforms and Unity.

Important

This library is currently in preview however all LINQ methods are already implemented. Currently supports .NET Stanard 2.1, .NET 8, .NET 9 and Unity. However .NET Standard 2.0 will also be supported when it is officially released.

using ZLinq;

var seq = source
    .AsValueEnumerable() // only add this line
    .Where(x => x % 2 == 0)
    .Select(x => x * 3);

foreach (var item in seq) { }
  • 99% compatibility with .NET 10's LINQ (including new Shuffle, RightJoin, LeftJoin operators)
  • Zero allocation for method chains through struct-based Enumerable via ValueEnumerable
  • Full support for LINQ operations on Span using .NET 9/C# 13's allows ref struct
  • LINQ to Tree to extend tree-structured objects (built-in support for FileSystem, JSON, GameObject)
  • Automatic application of SIMD where possible and customizable LINQ to SIMD for arbitrary operations
  • Fusion of my past LINQ (linq.js, SimdLinq, UniRx, R3) and zero alloc (ZString, ZLogger) impls

I aimed to create not just an experimental library but a practical one. It's also designed to handle high-load requirements, such as those found in games.

You can install it from NuGet/ZLinq. For Unity usage, refer to the Unity section. For Godot usage, refer to the Godot section.

dotnet add package ZLinq

ZLinq chains internally use the following interface:

public readonly ref struct ValueEnumerable<TEnumerator, T>(TEnumerator enumerator)
    where TEnumerator : struct, IValueEnumerator<T>, allows ref struct
{
    public readonly TEnumerator Enumerator = enumerator;
}

public interface IValueEnumerator<T> : IDisposable
{
    bool TryGetNext(out T current); // as MoveNext + Current

    // Optimization helper
    bool TryGetNonEnumeratedCount(out int count);
    bool TryGetSpan(out ReadOnlySpan<T> span);
    bool TryCopyTo(Span<T> destination);
}

Besides changing to a struct-based approach, we've integrated MoveNext and Current to reduce the number of iterator calls. Also, since structs automatically copy internal state, we've simplified the type complexity by unifying Enumerable and Enumerator(almostly types only implements custom enumerator).

public static ValueEnumerable<Where<TEnumerator, TSource>, TSource> Where<TEnumerator, TSource>(in this ValueEnumerable<TEnumerator, TSource> source, Func<TSource, Boolean> predicate)
    where TEnumerator : struct, IValueEnumerator<TSource>, allows ref struct

Operators have this method signature. C# cannot infer types from generic constraints(dotnet/csharplang#6930). Therefore, the traditional Struct LINQ approach required implementing all operator combinations as instance methods, resulting in nearly 100,000 methods and massive assembly sizes. However, in ZLinq, we've successfully avoided all the boilerplate method implementations by devising an approach that properly conveys types to C# compiler.

Additionally, TryGetNonEnumeratedCount(out int count), TryGetSpan(out ReadOnlySpan<T> span), and TryCopyTo(Span<T> destination) defined in the interface itself enable flexible optimizations. For example, Take+Skip can be expressed entirely as Span slices, so if the original source can be converted to a Span, Span slices are passed through TryGetSpan chains. For ToArray, if the sequence length can be calculated, a fixed-length array is prepared in advance, and operators that can write directly to the final array via TryCopyTo will do so. Some methods automatically use SIMD-based optimization if a Span can be obtained.

Gettting Started

Use using ZLinq; and call AsValueEnumerable() on any iterable type to use ZLinq's zero-allocation LINQ. Also, Range, Repeat, and Empty are defined in ValueEnumerable.

using ZLinq;

var source = new int[] { 1, 2, 3, 4, 5 };

// Call AsValueEnumerable to apply ZLinq
var seq1 = source.AsValueEnumerable().Where(x => x % 2 == 0);

// Can also be applied to Span (only in .NET 9/C# 13 environments that support allows ref struct)
Span<int> span = stackalloc int[5] { 1, 2, 3, 4, 5 };
var seq2 = span.AsValueEnumerable().Select(x => x * x);

LINQ to Tree

LINQ to XML introduced the concept of querying around axes to C#. Even if you don't use XML, similar APIs are incorporated into Roslyn and effectively used for exploring SyntaxTrees. ZLinq extends this concept to make it applicable to anything that can be considered a Tree, allowing Ancestors, Children, Descendants, BeforeSelf, and AfterSelf to be applied.

Specifically, by defining a struct that implements the following interface, it becomes iterable:

public interface ITraverser<TTraverser, T> : IDisposable
    where TTraverser : struct, ITraverser<TTraverser, T> // self
{
    T Origin { get; }
    TTraverser ConvertToTraverser(T next); // for Descendants
    bool TryGetHasChild(out bool hasChild); // optional: optimize use for Descendants
    bool TryGetChildCount(out int count);   // optional: optimize use for Children
    bool TryGetParent(out T parent); // for Ancestors
    bool TryGetNextChild(out T child); // for Children | Descendants
    bool TryGetNextSibling(out T next); // for AfterSelf
    bool TryGetPreviousSibling(out T previous); // BeforeSelf
}

Standard packages are available for FileSystemInfo and JsonNode. For Unity, it's applicable to GameObject and Transform.

FileSystem

dotnet add package ZLinq.FileSystem
using ZLinq;

var root = new DirectoryInfo("C:\\Program Files (x86)\\Steam");

// FileSystemInfo(FileInfo/DirectoryInfo) can call `Ancestors`, `Children`, `Descendants`, `BeforeSelf`, `AfterSelf`
var allDlls = root
    .Descendants()
    .OfType(default(FileInfo)!)
    .Where(x => x.Extension == ".dll");

var grouped = allDlls
    .GroupBy(x => x.Name)
    .Select(x => new { FileName = x.Key, Count = x.Count() })
    .OrderByDescending(x => x.Count);

foreach (var item in grouped)
{
    Console.WriteLine(item);
}

JSON(System.Text.Json)

dotnet add package ZLinq.Json
using ZLinq;

// System.Text.Json's JsonNode is the target of LINQ to JSON(not JsonDocument/JsonElement).
var json = JsonNode.Parse("""
{
    "nesting": {
      "level1": {
        "description": "First level of nesting",
        "value": 100,
        "level2": {
          "description": "Second level of nesting",
          "flags": [true, false, true],
          "level3": {
            "description": "Third level of nesting",
            "coordinates": {
              "x": 10.5,
              "y": 20.75,
              "z": -5.0
            },
            "level4": {
              "description": "Fourth level of nesting",
              "metadata": {
                "created": "2025-02-15T14:30:00Z",
                "modified": null,
                "version": 2.1
              },
              "level5": {
                "description": "Fifth level of nesting",
                "settings": {
                  "enabled": true,
                  "threshold": 0.85,
                  "options": ["fast", "accurate", "balanced"],
                  "config": {
                    "timeout": 30000,
                    "retries": 3,
                    "deepSetting": {
                      "algorithm": "advanced",
                      "parameters": [1, 1, 2, 3, 5, 8, 13]
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
}
""");

// JsonNode
var origin = json!["nesting"]!["level1"]!["level2"]!;

// JsonNode axis, Children, Descendants, Anestors, BeforeSelf, AfterSelf and ***Self.
foreach (var item in origin.Descendants().Select(x => x.Node).OfType(default(JsonArray)!))
{
    // [true, false, true], ["fast", "accurate", "balanced"], [1, 1, 2, 3, 5, 8, 13]
    Console.WriteLine(item.ToJsonString(JsonSerializerOptions.Web));
}

GameObject/Transfrom(Unity)

see: unity section.

LINQ to SIMD

WIP

Unity

The minimum supported Unity version will be 2022.3.12f1, as it is necessary to support C# Incremental Source Generator(Compiler Version, 4.3.0).

There are two installation steps required to use it in Unity.

  1. Install ZLinq from NuGet using NuGetForUnity
    Open Window from NuGet -> Manage NuGet Packages, Search "ZLinq" and Press Install.

  2. Install the ZLinq.Unity package by referencing the git URL

https://github.com/Cysharp/ZLinq.git?path=src/ZLinq.Unity/Assets/ZLinq.Unity

With the help of the Unity package, in addition to the standard ZLinq, LINQ to GameObject functionality becomes available for exploring GameObject/Transform.

using ZLinq;

public class SampleScript : MonoBehaviour
{
    public Transform Origin;

    void Start()
    {
        Debug.Log("Ancestors--------------");  // Container, Root
        foreach (var item in Origin.Ancestors()) Debug.Log(item.name);

        Debug.Log("Children--------------"); // Sphere_A, Sphere_B, Group, Sphere_A, Sphere_B
        foreach (var item in Origin.Children()) Debug.Log(item.name);

        Debug.Log("Descendants--------------"); // Sphere_A, Sphere_B, Group, P1, Group, Sphere_B, P2, Sphere_A, Sphere_B
        foreach (var item in Origin.Descendants()) Debug.Log(item.name);

        Debug.Log("BeforeSelf--------------"); // C1, C2
        foreach (var item in Origin.BeforeSelf()) Debug.Log(item.name);

        Debug.Log("AfterSelf--------------");  // C3, C4
        foreach (var item in Origin.AfterSelf()) Debug.
8389
Log(item.name);
    }
}

You can chain query(LINQ to Objects). Also, you can filter by component using the OfComponent<T> helper.

// all filtered(tag == "foobar") objects
var foobars = root.Descendants().Where(x => x.tag == "foobar");

// get FooScript under self childer objects and self
var fooScripts = root.ChildrenAndSelf().OfComponent<FooScript>(); 

Godot

The minimum supported Godot version will be 4.0.0. You can install ZLinq.Godot package via NuGet.

dotnet add package ZLinq.Godot

In addition to the standard ZLinq, LINQ to Node functionality is available.

using Godot;
using ZLinq;

public partial class SampleScript : Node2D
{
    public override void _Ready()
    {
        var origin = GetNode<Node2D>("Container/Origin");

        GD.Print("Ancestors--------------"); // Container, Root, root (Root Window)
        foreach (var item in origin.Ancestors()) GD.Print(item.Name);

        GD.Print("Children--------------"); // Sphere_A, Sphere_B, Group, Sphere_A, Sphere_B
        foreach (var item in origin.Children()) GD.Print(item.Name);

        GD.Print("Descendants--------------"); // Sphere_A, Sphere_B, Group, P1, Group, Sphere_B, P2, Sphere_A, Sphere_B
        foreach (var item in origin.Descendants()) GD.Print(item.Name);

        GD.Print("BeforeSelf--------------"); // C1, C2
        foreach (var item in origin.BeforeSelf()) GD.Print(item.Name);

        GD.Print("AfterSelf--------------"); // C3, C4
        foreach (var item in origin.AfterSelf()) GD.Print(item.Name);
    }
}

You can chain query(LINQ to Objects). Also, you can filter by node type using the OfType().

// get ancestors under a Window
var ancestors = root.Ancestors().TakeWhile(x => x is not Window);
// get FooScript under self childer objects and self
var fooScripts = root.ChildrenAndSelf().OfType(default(FooScript));

License

This library is under MIT License.

About

Zero allocation LINQ with Span and LINQ to SIMD, LINQ to Tree (FileSystem, Json, GameObject, etc.) for all .NET platforms and Unity.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C# 100.0%
0