ModEnc:Migration to structured data and forms

Due to games being computer programs and mods being modifications of said computer programs, there are many elements of the data stored at ModEnc that are very structured and uniform.

For example, every INI flag has a name and possible values it can take (Damage=15), and what values it can take, is part of a small set of answers.
There's a clear structure there:

• There are games.
• Each game supports specific INI files.
• Each INI file for this game supports specific flags.
• Each flag for this file for this game supports specific values.

Despite this, in the past, the data has never been stored in a structured way, and it hasn't been editable that way, either.

This page will act as a project page for a migration to more structured data storage and editing forms to change that.

Example of what we're going for

In preparation for the migration of the flag data, we have already migrated the data for metamods to the new system.
Metamods now have an "Edit with form" button, through which the data for a metamod can be edited in a much more direct and accessible way than editing templates.
This does not prevent templates from being used or the template from being edited, it is simply a quicker, easier option to achieve the same.

• 
  "Edit with form" button on metamods
• 
  form for editing metamods

In addition, the template is set up to store the data given to it in a structured way.
Where, previously, all metamods were simply independent pages that happened to look alike, the actual data is now being recognized and stored.
This gives us entirely new options for working with this data, like adding the latest release date of Phobos in the middle of a sentence (2025-12-10) or listing the developers of Ares (pd • DCoder • Electro • Renegade • Graion Dilach • AlexB), in a way that keeps this data updated when the data on the respective page is changed.
We can also quickly generate tables, like "show me all INI flags that require RockPatch":

_pageName	files	accepted values
AACombat	rules(md).ini	Boolean values: yes or no, true or false, 1 or 0
AITargetingType	rules(md).ini	Unsigned integers: All non-negative whole numbers from 0 to either 32767, 2147483647 or 4294967295.
AscentSpeed	rules(md).ini	Unsigned integers: All non-negative whole numbers from 0 to either 32767, 2147483647 or 4294967295.
Bounty	rules(md).ini	Unsigned integers: All non-negative whole numbers from 0 to either 32767, 2147483647 or 4294967295.
BountyAnim	rules(md).ini	Strings: Normal text.
BountyCostMult	rules(md).ini	Floating point values: Any decimal number (clearer range should be added in Template:Values).
DescendProximity	rules(md).ini	Unsigned integers: All non-negative whole numbers from 0 to either 32767, 2147483647 or 4294967295. (Limited to: Leptons)
DisableLoadScreenText	rules(md).ini	Boolean values: yes or no, true or false, 1 or 0
EliteBountyCostMult	rules(md).ini	Floating point values: Any decimal number (clearer range should be added in Template:Values).
EliteGetsBounty	rules(md).ini	Boolean values: yes or no, true or false, 1 or 0
EVerses	rules(md).ini	Percentages: Either a direct percentage (e.g. "50%"), or a floating point value (e.g. "0.5").
FourthBaseDefenseCounts	rules(md).ini	Unsigned integers: All non-negative whole numbers from 0 to either 32767, 2147483647 or 4294967295.
FourthCrew	rules(md).ini	Strings: Normal text.
FourthPowerPlant	rules(md).ini	Strings: Normal text.
GetsBounty	rules(md).ini	Boolean values: yes or no, true or false, 1 or 0
Hunter	rules(md).ini	Strings: Normal text.
IsIFV	rules(md).ini	Boolean values: yes or no, true or false, 1 or 0
IsNormalPlane	rules(md).ini	Boolean values: yes or no, true or false, 1 or 0
NewPlayableCountries	rules(md).ini	Signed integers: All whole numbers from -2147483648 to 2147483647; in rare cases, only from -32768 to 32767.
PlaceUrbanAreas	rmg(md).ini	Boolean values: yes or no, true or false, 1 or 0
PowerPlant	rules(md).ini	Strings: Normal text.
PowerPlants	rules(md).ini	Comma-separated list of strings.
SovietAdvancedPower	rules(md).ini	Strings: Normal text.
VeteranBounty	rules(md).ini	Unsigned integers: All non-negative whole numbers from 0 to either 32767, 2147483647 or 4294967295.
VeteranBountyCostMult	rules(md).ini	Floating point values: Any decimal number (clearer range should be added in Template:Values).
VeteranGetsBounty	rules(md).ini	Boolean values: yes or no, true or false, 1 or 0

Questions

Fundamentally, both changes, the introduction of structured data and the introduction of forms, will make ModEnc easier to edit and allow greater flexibility in the use of its data.
However, some questions arise that we would like your opinion on the talk page on:

Strategic Changes

If we're updating the core data of ModEnc, we might as well ponder what data we even have and why.
If we're altering the way we're keeping, editing and presenting ModEnc's data, now is a good time to ask: What data do we want to keep, edit and present on ModEnc?

Game Focus

Released Games

The current flag header includes options for Generals, Zero Hour and Kane's Wrath.
These games never really had content on ModEnc and neither did their antecessors.

The same is true for Tiberian Dawn and Covert Operations: While -at the time of this writing- there are two flags which claim to be available in Tiberian Dawn, this is likely to be an error, considering TD was not INI-based.

• Unload

Nyerguds had a project running to make TD accept INI-based modification, but even if that project goes somewhere, the documentation would not pertain to Tiberian Dawn, but to Nyerguds's metamod.

And we don't have a lot of data on the game pages, either.
All the game info is stubs and the only useful info are command line arguments which may well be on pages for ra2.exe and ra2md.exe instead of Red Alert 2 and Yuri's Revenge

It might be best to declare a clear focus on Rules(md).ini-based games, remove everything else from content, and to redirect the game pages to C&C Wikia.

Open Source Games and Clones

On the flipside of that, with Red Alert being open source now, there might be forks generating new flags in the future.

There have also been clones like OpenRA for a long time.
OpenRA uses YAML, which is similar enough to INI that we could reasonably support it if we wanted to.

As we get rid of support for games wholly dissimilar to RA/TS/RA2, we may want to add support for games that are similar enough for there to be value in having them.

Metamods

The Phobos-people notoriously hate us enough that they would rather clone us than talk to us, but there might be other Ares-clones and metamods out there who might be interested in having their flags documented on ModEnc.

As we open up the flag template to allow any dependency to be declared, adding those projects will be easier than ever.

The only questions are: What is out there and what is worth cataloguing?

Metamod-Documentation

Since we already have the flags for a metamod and the installation instructions are usually very similar ("run this program to alter the exe", "run Syringe with this DLL", "run this launcher"), the full documentation for any given metamod is likely not dramatically bigger than what we would already have.

It might be worth having the rest of the documentation here as well and then autogenerating grouping it here for easy consumption and exporting it to PDF for download.

This would also prevent issues like with Ares's documentation, where maintainer-abandonment leaves the documentation unaltered for years.

Mods

Mods have always been listed on ModEnc, but back in the day, we had the same notability-problem Wikipedia has: If you allow everything, then you get 1000 dead pages for mods that were enthusiastically started and then abandoned a week later.
These days, the community is small enough that it should be possible to generate a fairly stable list of mods that exist and are still maintained.

So we could go in the other direction and not just say "mods can be made like this" but also say "these are mods that have been made".
In addition to saying "you can make fancy mods with Ares", we can say "here are some mods made with Ares".

This would, of course, require someone to actually maintain that data.

There was an idea back in the day for LaunchBase, which we could adapt if we had the mods in structured data: By setting up a fixed URI that displays the structured data as JSON, a mod manager could pull a list of mods and metamods with download locations and dependencies from ModEnc, to then offer automatic download and activation, like LB did.

The original vision involved the mods having microdata on their websites, to make their mod information parseable by such a directory.

If mods bought into this, ModEnc could query microdata from mod sites, automatically update the core data of the mod on ModEnc, and then generate a complete, parseable mod directory for mod managers.

Then again...are there enough people left who play more than one mod?

Page Separation

In terms of pure data organization, the proposed solution of page splitting obviously has strategic implications.

Challenges

Data Connection/Normalization

Problem

After twenty years of new ideas and developments being tacked on, the current iteration of Template:Flag is kind of terrible in the way it expects and accepts data.

For example,

• it has parameters to mark the flag as belonging to 15 different games, addons and metamods
• it has three parameters to mark the flag as requiring a specific metamod
• it has three parameters to mark the flag obsolete in a specific game

Here's the thing: These are not cohesive sets.
e.g. you can define a flag as an Ares flag either by setting ares=yes, or by setting aresflag=yes, which is semantically the same most of the time, but not always.
You see, the former says "this flag is available in Ares", while the latter says "this flag is available because of Ares".
Compare the following ways of communicating flag availability:

yr=	ares=	aresflag=	result
	yes		Flag exists only in Ares
		yes	Flag exists only in Ares
yes		yes	Flag exists only in Ares
no		yes	Flag exists only in Ares
yes	yes	yes	Flag exists only in Ares
yes	yes		Flag exists in Ares, but not exclusively
yes	yes	no	Flag exists in Ares, but not exclusively
yes			Flag exists in Ares, but not exclusively

There are "game" parameters for HyperPatch and Ares, but not for RockPatch and Phobos.
There are "introduced by" parameters for RockPatch, HyperPatch and Ares, but not for Phobos and Vinifera.
The "introduced by" parameter for RockPatch is called "rponly", while for HyperPatch, it is "hpflag".

There are three parameters to declare a flag is obsolete, but only for TS, RA2 and YR. Not for any other games, not for any metamods.

It's a wholly inconsistent mess of technical debt.

Possible Solutions

Independent Storage of Availability

The current idea is to have this replaced with independent engine availability markers, which basically say "this flag is available in X as of version Y".
So if something is an Ares flag, you would simply declare it to be available in Ares as of Version 1.12.
If it is available since TS 1.20, you would simply add more definitions:

• This flag is available in Tiberian Sun as of Version 1.20.
• This flag is available in Firestorm as of Version 1.0.
• This flag is available in HyperPatch as of Version 1.0.
• This flag is available in Vinifera as of Version 1.0.
• This flag is available in Red Alert as of Version 1.0.
• This flag is available in Yuri's Revenge as of Version 1.0.
• This flag is available in RockPatch as of Version 1.0.
• This flag is available in Ares as of Version 1.0.
• This flag is available in Phobos as of Version 1.0.

This way, it is very clear where a flag is available: If your product is in the list and if you have a version equal to or higher than the version listed, that flag is available to you.

Each availability marker would carry "obsolete" as a possible parameter, so you could do things like

• This flag is available in Red Alert as of Version 1.06.
• This flag is available in Counterstrike as of Version 1.0.
• This flag is available in Aftermath as of Version 1.0.
• This flag is available in Tiberian Sun as of Version 1.0, but obsolete.

The formatting would obviously more compact.

Disadvantages

• The prime disadvantage of doing it this way is that the availability data is no longer part of the flag template. Data-wise, it's a separate piece of data, and editing-wise, it's a separate template.
  So instead of marking on the flag template that this is a YR flag, the editor has to add a tiny separate template for each engine the flag is available in.
  However: Depending on the way it is modeled, this separate template can be added to the editing form, so it doesn't look separate to the editor
• Because the availability data is stored separate from the flag data, querying flags by availability on the basis of the structured data will require a join.

Advantages

• Technically, taking the availability data out of the flag definition is correct: "What is this flag?" is a different question than "Is this flag available to me?".
• Separating the flag availability data from the flag data means new projects (like Ares-forks) can be added to the page immediately, without requiring changes to the template
• Separating the flag availability data from the flag data enables much more uniformity and flexibility in terms of how version data is handled - the same data is stored the same way for all flags and all engines, and can be adjusted when necessary.
  This means projects like Vinifera or Phobos will be treated equal to projects like HyperPatch or Ares on flag pages.

Multiple Definitions of the Same Flag

The "Edit by form"-feature is designed for uniform pages of the same kind. The typical "here's a sidebar with the data" type of pages.
It makes the assumption that there's a type template for this type of page, and that each instance of that type has that template at the top of the page.

In other words: Editing by form only works for Template:Flag at the top of the page.

This collides with the way ModEnc is currently set up, in that we have pages like Image, where Template:Flag is included thrice, once for each context the flag is available in.
This is more readable and easier to understand than squishing all the data for all the use cases into one template and forcing the reader to decipher which parts are applicable to their current situation.

Editing a page like that with a form would only edit the first instance of the template, and move it from down in the first section to up at the top of the page - i.e. destroy the layout.

Possible Solutions

Splitting the Pages/Going Uniform

The simplest solution would be to not have pages in which the same flag is described in different contexts.
There is a quasi-philosophical question behind this in terms of "what is the identity of a flag?".

Currently, ModEnc is set up as if a flag is the same flag if its name is identical.
Image is Image is Image, it can just be used in different contexts.

However, Image on Animations in artmd.ini takes a piece of text to determine the file name of the animation.
Image on Tiberiums in rulesmd.ini takes a number to look up an index of the OverlayTypes array.

Yes, in both cases the idea is to determine what The Thing looks like, hence the identical name, but you could easily have named Image on Animations FilenameFrom and on Tiberiums IndexConstant, and no one would ever have considered them the same flag.

It stands to reason that we could put them on different pages without violating the soul of Image.

Disadvantages

• Finding some flags will be less direct. We will have disambiguation pages and "this page is about Image on Tiberiums, if you're looking for Image on Animations ..." kind of headers on a number of pages.
  This is unfortunate, as we're currently very good at having a "go to the flagname as the pagename and find information" kind of workflow, but it's realistically not dramatic.
  People will end up on an overview page and can select what they want from there.
• Not having the flag information on a page equal to the page name will lead to more cases where we have to make the name of the flag explicit, so that it's clear that the flag is named "Image", not "Image (Tiberiums)".
• This solution inevitably requires some renovation in terms of splitting up pages, moving text around, etc.
  It'll also break links insofar as that links to Image#Tiberiums will only link to a friendly note that this content was moved to Image (Tiberiums).

Advantages

• Forms will work as intended.
• Better, cleaner and more uniform organisation of flag descriptions in context: Rather then ending up at Image and having to scroll and to check and to see what applies to your use case and what doesn't, you end up at Image (Tiberiums) and know that everything you're looking at pertains exclusively to the case you're researching.
  And all the data is in the same place: Rather than having to scroll down to find the flag header for Image on Tiberiums, it's right there at the top as it is for all other flag pages.
  In terms of showing the same data the same way for each application of a flag, this is the superior solution.

Splitting Data Input and Data Output

It is possible to have templates that can be used multiple times and have them editable in the form, but they'll still appear at the top of the page.
We could work around the problem by having one primary data entry template, and a secondary, tiny data display template.

We would a FlagData template, that would receive flag name, file name, section name, etc., which could appear multiple times at the top, be editable by the form, but display nothing on the page.

We would then have a separate FlagDisplay template which would be added as a one-liner at the top of a section to display the appropriate header.

{{FlagData
|Name=Image
|Files=Rules.ini
|Section=Tiberiums
|blablablablabla
}}
{{FlagData
|Name=Image
|Files=Art.ini
|Section=Animations
|blablablablabla
}}

==on Tiberiums==
{{FlagDisplay|Name=Image|Files=Rules.ini|Section=Tiberiums}}

Text about Image on Tiberiums

==on Animations==
{{FlagDisplay||Name=Image|Files=Art.ini|Section=Animations}}

Text about Image on Animations

This will be doable because the data template will store the data as structured data, and the display template can retrieve that data anywhere.

Disadvantages

• Separating input and output like that is not what templates and forms are designed for.
• Will look a bit more messy, because the data templates are not where the data is shown, there is data for multiple use cases in one place, and everything will rely on more arguments to know their context.
  On the other hand, if editing by form, you shouldn't see the mess.

Advantages

• Forms will work as intended.
• All the hard data is at the top of the page when editing.
• Maintains the organizational status quo in terms of pages and page contents

Acceptance

We could, hypothetically, just accept that it doesn't work for all cases.
A significant majority of cases on ModEnc conform to the ideal of "one instance of Template:Flag on the page, at the top".

We could just create the new template with a focus on those pages, put a warning on the template and the form saying that it'll only work for the primary instance of the flag template, and then create a secondary instance of the template that is intended for follow-up usage without form integration.

{{FlagPrimary
|Name=Image
|Files=Rules.ini
|Section=Tiberiums
|blablablablabla
}}

==on Tiberiums==
Text about Image on Tiberiums

==on Animations==
{{FlagSecondary
|Name=Image
|Files=Art.ini
|Section=Animations
|blablablablabla
}}

Text about Image on Animations

Disadvantages

• Forms do not work as intended, since not all instances of the template are editable by form.
• Inconsistent template application: Users have to use the right template in the right circumstances.
• Inconsistent editing: To update two instances of the same flag template on the same page, you edit one via form and one via text.
  Or you forego the form, at which point everything was for naught.

Advantages

• Simplest solution: We're avoiding the effort of correction by not correcting anything.

Conclusion

As you can see, there are various things to consider and we would love to hear your opinions and suggestions on the talk page.