RP9 Manifest Examples and Test Cases
This document lists examples, corner cases and other test scenarios that may be of interest to software developers who wish to support RP9 files, with special consideration to user-edited description and configuration data.
RP9 files were first introduced in the 2009.0 versions of Amiga Forever and C64 Forever, which were based on version 2.0 of the RetroPlatform Player framework. Version 2.2 of RetroPlatform Player extended the original RP9 specification to include user-editable description and configuration data (<description>, <configuration>, <media> and <startup> elements). These new components are part of the 2009.2 versions of Amiga Forever and C64 Forever, and of newer versions.
RetroPlatform Player is not only a playback tool, but it also features RP9 Toolbox, which can import, export and rescan RP9 files, and the RP9 Title Editor, to individually edit files. The rescan operation regenerates the configuration data by comparing the RP9 media content with data stored in RetroPlatform Library (a database of known content). The editor also has an autoconfiguration option (applied to the current file) which is similar to the rescan procedure.
RetroPlatform Player and RP9 Toolbox comply with RP9 design goals which include:
- RP9 files should not be unnecessarily different if they contain substantially the same data (exporting from RP9 to individual files and back to RP9 should produce a file identical to the original, if the same RetroPlatform Library was available for automated content recognition)
- Manifest data generated by newer RetroPlatform Library should not be overwritten on the basis of older data (the RP9 ecosystem should improve over time, not get worse)
- Configuration data that was manually created by a user should generally be respected, unless the user expressly decides (e.g. by disabling the "Protect user-modified configuration tags" option in RP9 Toolbox) to rescan the file, and newer RetroPlatform Library data is available for that file
Achieving some of these goals was not trivial. For example, a simple export and import of an RP9 file set results in a new XML manifest file being written, with a new (current) file timestamp. Because RP9 files are ZIP-based containers, their content also changes to reflect properties such as file dates. RP9 Toolbox overcomes this by setting the manifest file date to the same date of the newest file in the set (e.g. an existing disk image file).
While the up-to-date description and configuration data provided by RetroPlatform Library helps RetroPlatform Player implementations to accurately visualize and play back RP9 content, third-party players can use RP9 files without loading additional data. The RP9 file name and the XML manifest respectively contain sufficient description and configuration data for most playback purposes.
With respect to third parties, RP9 design goals include:
- Adoption of the RP9 format by third parties is encouraged for all purposes, including editing and playback
- RP9 files should be playable on third-party players even without RetroPlatform Library
XML Manifest: Required Elements
The following elements are required in the rp9-manifest.xml file.
This is the outer container element.
This element contains <host> and <playerversion>. It is essential for graceful forward-compatibility notifications to the user (e.g. required updates).
If this is missing, RetroPlatform Player will show an error message.
May be safely set to a minimum version of 18.104.22.168, which is supported by all generations of RetroPlatform Player (including the oldest ones, via free updates). Version 22.214.171.124 supports all CBM systems as in C64 Forever 2009-2012, and the Amiga 1000, 500, 600, 1200, CDTV and CD³² as in Amiga Forever 2009-2011.
Support for new Amiga systems (Amiga 2000, 500 Plus, 3000, 4000) was introduced in version 126.96.36.199, which is required to run configurations based on these systems.
Version 188.8.131.52 added support for new CBM systems: C128 starting in 80-column mode, SX-64, CBM 510, CBM 720, CBM 232, and CBM V364.
The version number should be increased with extreme care, because it indicates an absolute requirement. If the version of the player application opening the file is older than the indicated version, the player displays a message indicating that a newer player version is required.
This element contains all content-specific elements (description, configuration, media, etc.), as well as the OID attribute.
This element may contain a copy of configuration data sourced from RetroPlatform Library, or heuristic data, or user-edited data.
As a minimum, it must contain the <system> element, which allows to use a system in its canonical configuration (including default drive, mouse, and joypad options).
XML Manifest: Optional Elements and Attributes
The following elements and attributes are optional in the rp9-manifest.xml file. This is not an exhaustive list, but merely a selection of what is more relevant to the topic discussed here (playback scenarios and user-edited data).
<application oid="184.108.40.206.5" libraryversion="220.127.116.11" score="100" user-edited="true">
The oid attribute indicates the unique application OID (always inclusive of application variant), based on RetroPlatform Library data (the above "18.104.22.168.5" is only a fictitious example). It helps applications that have access to RetroPlatform Library data access additional, accurate and up-to-date description information, and configuration information (unless the RP9 file contains newer or user-edited configuration data). The oid is also required for purposes such as associating "star rating" data, and for keeping track of saved states.
The libraryversion and score attributes indicate the RetroPlatform Library version that was used to identify the application, and the score with which the application was recognized (100 indicates an exact match). These attributes help express the version and the quality of the data, so that, for example, it is not overwritten by older data.
The optional user-edited attribute is set in the <application> tag whenever the data in the element is created or modified manually. This helps appreciate and protect user contributions and preferences (both for playback purposes and during rescans that might otherwise reset all data). If the attribute is missing, it is assumed to be False.
If the data does not come from RetroPlatform Library, the oid, libraryversion and score attributes are omitted. If the data comes from RetroPlatform Library and is later edited, the attributes should be retained (adding the user-edited attribute), unless the edited content references a different title.
The optional catalog-only attribute may be set in the <application> tag in RetroPlatform Library to mark records that have no configuration data, but are only present for identification and cataloging purposes. The attribute is never used inside RP9 files.
The libraryversion attribute of the <application> tag is required whenever the oid attribute is set. If the tag is missing, the value is assumed to be 1.0.
Within RetroPlatform Library, <description> data contains information for visualization and presentation purposes, not for playback (configuration) purposes. As of 2.x versions of the RetroPlatform framework, this element contains a subset of the description data that is otherwise stored in RetroPlatform Library for display purposes (i.e. not for configuration or playback purposes), including some information such as the publisher name and application title that is usually also found in the RP9 file name (where however it may be damaged by file naming constraints).
This element references the media images (e.g. disks or tapes), if any, used by the configuration.
Playback - Common Scenarios
For applications that have access to RetroPlatform Library data (e.g. RetroPlatform Player implementations like Amiga Forever and C64 Forever):
- If the application OID is present and found in the available RetroPlatform Library data, look up the description and configuration information in RetroPlatform Library
- If the application OID is missing or unusable, scan the media to recognize it, and see if an OID can be generated for a temporary lookup (the RP9 is not modified)
- If an application OID is still unavailable after the previous steps, extract the publisher, application title and other details from the RP9 <description> element, or, as a last resort, from the file name
- If user-edited="true", or if the "libraryversion" attribute references a library newer than the one that is available to the application, always use the description and configuration from the RP9 data (if the RP9 contains no <configuration> element, try to look up the configuration from RetroPlatform Library, even if it is a "catalog-only" entry that only has the <system> element)
- In all other cases, use description and configuration data from RetroPlatform Library, or heuristic configuration
For applications that do not have access to RetroPlatform Library data (e.g. third-party players):
- Preferably extract the publisher, application title and other details from the RP9 manifest data
- As a second-best choice, extract the publisher, application title and other details from the RP9 file name
- Use the configuration from the RP9 manifest data
Playback - Special Cases
- Missing <playerversion> or <host>, or <playerversion> referencing a version newer than the current player implementation version: blocking condition
- "oid" attribute present, but missing "libraryversion" attribute: take description and configuration from RP9 file (it is up to the rescan to regenerate/repair the data, if so desired)
- "oid" attribute present, but "libraryversion" references a library that is newer than the one installed: treat as if "libraryversion" is missing and display warning message encouraging to check for library updates (with "Do not show again option")
- "oid" attribute missing, or "oid" present but no match found in library (yet also no reference to a newer "libraryversion", as in the previous case): scan the media to recognize it, and see if an application OID can be generated for a temporary lookup (the RP9 is not modified)
- "user-edited" attribute set: always use description and configuration data from RP9 elements (not from RetroPlatform Library, regardless of versions - if the user wishes to override this, the RP9 must be edited or rescanned in RP9 Toolbox with the desired "Protect user-edited configuration tags" setting)
Rescan - Common Scenarios
A rescan operation rebuilds the OID, <media>, <description>, <configuration> and <startup> data in RP9 files based on (ideally) the most current RetroPlatform Library data. Content recognition is attempted by media and/or by description. Information in the <addon> and <extras> elements is preserved and not modified. The version information in the RP9 file helps to make sure that newer data is never overwritten by older data. The media in the RP9 file is always rescanned, even if the RP9 contains an existing application OID reference.
As user-edited data is marked as such ("user-edited" attribute), it is also possible to choose whether to preserve such data. In RetroPlatform Player implementations like Amiga Forever and C64 Forever an option in RP9 Toolbox named "Protect user-edited configuration tags" makes it possible to choose whether older RP9 configuration data (based on the "libraryversion" attribute) may be overwritten by newer RetroPlatform Library data, if such data is available. This is generally acceptable, because individual RetroPlatform Library data changes originate from expert users and, over time, passes additional verifications and reputation-based mechanisms.
In all cases:
- If the application performing the RP9 rescan has an older version of RetroPlatform Library than the one referenced in the RP9 file, then no changes are applied to that file (if it is the same version, the rescan takes place), because it is assumed that the RP9 already contains newer data
- Lack of version information in the RP9 <application> tag is always treated as "older version"
If "Protect user-edited configuration tags" is False:
- If the media is recognized, the RP9 elements are rebuilt, with "oid" and "libraryversion" attributes referencing the current RetroPlatform Library; any user-edited data is reset with library-sourced data, and the "user-edited" attribute, if present, is cleared
If "Protect user-edited configuration tags" is True:
- If the "user-edited" attribute is set, do nothing; otherwise, proceed as when "Protect user-edited configuration tags" is False
An RP9 file may reference a default configuration (e.g. a C64) with no media. Or it may reference a configuration (e.g. an Amiga 600 HD) with the default system hard drive (in a shared location outside of the RP9).
In these cases, if the RP9 contains no media (but for example it only references shared system media), then a Rescan operation will only use the application OID to identify the application. If neither media nor an application OID are set, then the Rescan will not attempt to update any data.
An Autoconfigure from Files function is more specific in intent, and should not do anything if there are no media files (even if there is an OID).
"catalog-only" Library Entries
The optional "catalog-only" attribute may be set in the <application> tag in RetroPlatform Library data to mark records that have no configuration data (other than the <system>) and no cataloged media, but are only present for identification and cataloging purposes. The attribute signals that the absence of <configuration> (other than <system>), <media> and <startup> data is not a feature of the title, but rather it is due of the nature of the entry, meant for cataloging purposes, rather than for configuration purposes. The attribute is never used inside RP9 files.
In playback and rescan scenarios, the presence of the "catalog-only" attribute in the RetroPlatform Library data has an effect similar to the "user-edited" attribute in an RP9, i.e. it makes any <configuration>, <media> and <startup> data in the RP9 prevail over the RetroPlatform Library data, ensuring that the lack of such data in RetroPlatform Library does not have a tombstoning effect if the data is present in the RP9. More specifically, in a rescan scenario the presence of the "catalog-only" attribute in the RetroPlatform Library data ensures that existing RP9 manifest data is never stripped of valid <configuration>, <media> and <startup> data that it may contain. A <system> element (in the <configuration> element) in the RP9 prevails over the same RetroPlatform Library data, even if the latter has a newer version (otherwise there could be a conflict with the other RP9 manifest elements).
Editing and Authoring
When the RP9 Title Editor opens an RP9 file, it uses the same steps employed by the playback procedure to determine the configuration, with a prompt to choose between data in the RP9 file and RetroPlatform Library data, if newer data is available.
- If data is modified, the RP9 Title Editor also sets the "user-edited" attribute.
- If the original RP9 had a publisher OID, and the publisher name is edited, both the publisher OID and the application OID are cleared.
- If the new publisher name is selected from the list of known publishers, a new publisher OID is set (and the application OID is cleared, if present).
- The original application OID is cleared if either the publisher name or the application title are edited.
- A new application OID is set only if the RP9 being edited has a publisher OID, and if an application is selected from the list of known applications for the selected publisher and platform.
- When a new application OID is set during RP9 editing, a default ".1" application OID variant suffix is set internally. The variant has no effect for configuration purposes (all variants share the same configuration), and if the edited data is submitted for RetroPlatform Library improvement purposes then the variant needs to be redefined in a unique way anyway.
- The RP9 Title Editor always displays the application OID with the trailing variant number.
Checklist for Authoring Tools
- Make sure you that you consider the common playback and rescan scenarios, and the special cases
- If your application allows the user to edit the RP9 data, it should preserve the original application "oid" and "libraryversion" information, and set the "user-edited" attribute; if the application "oid" attribute itself is reset, "libraryversion" may be omitted if the application does not use data compatible with the RetroPlatform Library versioning system
- Test playback of your custom RP9 files with a RetroPlatform Player implementation, ideally also doing a rescan and comparing the original and updated manifests
Amiga Forever supports an additional feature compared to the RP9 playback and rescan scenarios described so far, namely legacy support for the original RP2 files introduced in 2008 and used until 2009, when version 2.2 of the RetroPlatform Player framework introduced user-editable description and configuration data embedded in the RP9 file. "RP2" was renamed to "RP9" in 2009 shortly before the introduction of the 2.2 framework. Previous-version RP2 and RP9 content files only had an OID in the <description> element and no <configuration> or <media> elements.
Version 2.2 of the RetroPlatform Player framework was introduced in Amiga Forever 2009.2. Since then, all version of Amiga Forever (i.e. including the free updates available to the original pre-2.2 users) still offer the following functionality for previous-generation files:
- Automatic conversion to the new RP9 format (if RP2 files are found in the Games and Demoscene player folders)
- Manual conversion via the Rescan feature of RP9 Toolbox
It is safe to link to this page.