Deprecations and Updates (MfGames .NET Libraries)

It's been a while, but we had a productive week making changes to the library.

Deprecations

One advantage of having a monorepo of all of the MfGames packages is being able to link everything together without doing a railcar shuffle when one package requires another and another and takes half a day to update a dependency. The drawback is that it makes it harder to abandon one of the libraries. Today is one of those days where we realize there are three libraries that need to be dropped from active development.

• MfGames.Identifiers was a good idea at the time but then we found that other libraries can handle the same goals and we don't have to maintain ours. Also, working with source generated files is painful so we are thinking those libraries are a better choice for now.
• MfGames.Nitride.SpectreConsole was an attempt to funnel Serilog output through SprectreConsole so we could get progress bars playing well with output for MfGames.Nitride. It ended up being cumbersome and, while it worked, it also had some minor quirks that were irriating enough that we just gave up on the idea of progress bars entirely instead of trying to fix it.
• MfGames.Serilog.SpectreExpressions was the underlying library for the MfGames.Nitride.SpectreConsole. This ended up being a mirror of the existing expressions library from Serilog because the original was sealed and couldn't be extended without copying the wheel. It also didn't have the full ability as the original library because it was hard to maintain code that isn't ours while needing to track any vendor changes. So when MfGames.Nitride.SpectreConsole was deprecated, so was this.

Now, we don't want to abandon old code entirely, so we moved those projects from the //src directory into the //attic/ directory. That way, the code remains available in case we find another use of that. This also means that those projects will “bit rot” in that they won't be refactored and maintained with the code in //src/, //tests/, and //examples/. The more time passes on, they will become less useful as code until we clean out the attic, which may be years from now.

Package Updates

In addition, we've done a round of updates to keep everything running smoothly and handle some vulerability warnings in our dependencies. Overall, this was a simply process except libraries dealing with:

• CLI
• MfGames.Nitride.IO

For the CLI, we use System.CommandLine as the base of our MfGames.ToolBuilder library, which is basically an opionated setup that uses Serilog, Autofac, and dependecy-injected commands. When moving from version 2.0.0-beta8 (it was in beta for over five years) to 2.0.9, there were some drastic refactoring that needed to be done. This resulted in all the CLI commands (MfGames.ToolBuilder and MfGames.Nitride) having a breaking change since the signatures changed across the board. There were relatively few dependencies on this, but they were all bumped to major versions.

For the other libraries, this was a feature (0.x.0) update because of potential side effects we haven't been able to identify.

MfGames.Nitride.IO

The other change was MfGames.Nitride.IO. In specific, we changed how the ReadFiles operation worked because of real-world difficulties. Most of our IO operations are based on the excellent library, Zio which gives use UPath which simplifies path processing, virtual file systems which makes test writing a lot easier, and remapped structures that let us treat a folder as /sources/allegro as / so our Fedran can treat each project Git repository as a root folder, which simplifies logic. We also like using recursive globbing (ones that use ** to recurse into dirctories in addition to * for matching at a given level).

The problem is that Zio doesn't support globbing natively. So our original implementation was to iterate through every file using Zio, then compare it against the glob.

For simple cases, this was fast. However, when we're doing a search like /src/**/chapter-*.md in a project directory, this also processed through /node_modules, .direnv/, and target/ folders which have a staggering amount of files that we don't want.

Nowe, we have both include and exclude patterns in an ordered list. For each one, we parse the pattern and attempt to do some simple heuristics with the root comparisons before we devolve into the “grab everything to compare it”. What this means that for /src/**/chapter-*.md, we go into /src before we grab everything. The other massive folders are looked at, and the code chooses not to go into it. If there are no ** in the pattern, this will give us a rather prescise scan which is faster than before.

// Old version
ReadFiles readFiles = Resolve<ReadFiles>()
  .WithPattern("/chapters/chapter-*.md");

// Old version without using the .With* methods.
ReadFiles readFiles2 = Resolve<ReadFiles>();
readFiles2.Pattern = "/chapters/chapter-*.md";

// New Version
ReadFiles readFiles = Resolve<ReadFiles>()
    .AddIncludePattern("/chapters/chapter-*.md");

To keep the conventions from .gitignore and other files, if you give the pattern as *.txt, we change it to /**/*.txt.

Which leads to the exclude patterns. /** effectively goes back to the old code of grabbing everything and comparing it. But if you then add an exclude pattern (ideally without a **), it will stop processing the includes if it already knows it will be excluding it.

ReadFiles readFiles = Resolve<ReadFiles>()
    .AddIncludePattern("/**/chapters/chapter-*.md")
    .AddExcludePattern("/node_modules")
    .AddExcludePatternList(
        "/.direnv",
        "/target");

This will eventually lead up to being able to pass in .gitignore directly into the code, but we want the code base to settle (e.g., use it for awhile) before we add that.

There is also a .WithPatternList() which takes a list of a typesafe collection, but that's a more edge case and if you need that, then you might want to look at our tests for examples of how to write those.