Just a week ago, CDISC released version 1.0.0 of the CORE validation software, followed by a webinar last Tuesday.
CORE stands for "CDISC Open Rules
Engine", and, as an open source project, can be used by everyone without any limitations (MIT license). CORE must be seen as a "Reference Implementation" meaning that everyone can make something similar, in any program language, and call it "CORE compatible", as long as the results are identical to that of the reference implementation. CORE v.1.0.0 itself has been
developed using the Python programming language.
CORE v.1.0.0 is the result of a lot of work at CDISC, not forgetting the great work of many volunteers who contributed to the project, either as rule implementers (see below), reviewers doing QC, or improving the source code.
The rules themselves have been implemented starting from the published CDISC rules in Excel, and the FDA business rules as published by the FDA. For the latter, a lot of communication with FDA was necessary, as the "business rules" are often defined at a "high level", which mean that they could be open for interpretation. Therefore, in cooperation with the FDA, many of the FDA business rules needed to be split in different single rules.
All the rules have been developed in YAML, this, as YAML files by the volunteers. This must have been thousands of work-hours. The CORE engine than translates
these YAML files into JSON, which is then picked up by the engine and executed using the Python code. The reason that YAML was chosen was that most of the volunteers are matter experts, not programming experts, so we needed something
that was easy to use.
All these YAML rules can be found (as everything is Open Source) on the "CORE Rules Github website". They are very easy to
read and understand so that every non-programmer can understand them.
For example:

About one-and-a-half year ago (May 2025) a cooperation between Verisian and CDISC started, in order to boost the development of CORE. This cooperation has been very successful, also as Verisian strongly believes in the value of open source. This cooperation has led to a large common code set (written in Python), with exactly the same set of rules, but some differences in the engine itself. So one can speak of the Verisianimplementation as a "fork" of the CDISC implementation. Both engines do however deliver exactly the same results when applied on the same set of data.
So, what is covered?
Both CDISC and Verisian have stated that this v.1.0.0 release only contains all of the CDISC SDTM rules and the FDA business rules. This is not entirely correct. As I found out,
both also contain some (i.e. incomplete set) of the TIG (Tobacco Implementation Guide) rules, and the SEND rules for non-clinical submission datasets that are overlapping with an SDTM or TIG rule. As said, these are however incomplete, and work is underway
to get these completely done - we need more volunteers for reviewing and QC-ing. Also a number of the USDM rules have already been implemented, but I haven't checked on that yet. . The latter uses JSONata,
which is similar in approach with Schematron, but than for … JSON. This is very interesting, as several people have already implemented all the Define-XML rules in Schematron, so it should not be too hard to translate these into JSONata and thus implement them in CORE.
The rules also contain rules that validate SDTM/SEND content against the define.xml, but not the define.xml specific rules. All together, over 1400 rules have been implemented.
We did already implement CDISC CORE v.1.0.0 in most of our systems. As we also had implemented earlier versions of CORE, this was a matter of hours. So you can expect new release of some of our software offerings, like the SDTM-ETL mapping software, pretty soon.
I already said a few words about the Verisian implementation of CORE v.1.0.0. So, many of you will ask what is the same and what is different.
Let us first look as what is the same:
First of all, the set of rules is completely identical, which already mostly guarantees that the results will always be the same as with the CDISC reference implementation.
Furthermore, both allow to extend the CORE engine with "custom rules" and group them into "custom standards". I already wrote about that in the past. See the articles "Generating custom CORE Rulesstarting from CDISC SDTM Dataset Specializations" and "CDISC CORE and Custom Rules and Standards".
All the CLI (Command Line Interface] commands are identical too, except that the Verisian version has a few more, but these are mostly additional "list" comands.
What is then different?
The most important difference is about how the data is stored internally.
CDISC CORE uses Dask instead of Pandas when dataset size exceeds 1/4 of available RAM. One can however force the CDISC CORE engine to use Dask all the time by setting the memory threshold to 0. Dask has the advantage that it can handle datasets that are larger
than what can fit in memory, also allowing parallel processing across multiple CPU cores or even multiple machines. This should improve the performance for large datasets.
Verisian choose a different approach: the engine stores the data it validates in a PostgreSQL database while it runs. The engine already comes with a build-in PostgreSQL database, so the user does not need
to install one. The user (or the organization that sets up the Verisian engine) can however decide to set up one when it is advantageous to keep the data in the database between runs. With the build-in database, the data is
removed from the PostgreSQL database after each run.
Essentially, the use of a database allows to validate very large to huge datasets, which is of course interesting for sponsors. For development work, such as when
using our SDTM-ETL the CDISC version can however be sufficient.
For implementers, an important difference is that the Verisian implementation requires Python (exactly said, Python version 3.12) must be installed: it doesn't come with precompiled Windows, Linux or Mac executables. In the CDISC implementation, such precompiled executables are available. I think this is an advantage for single-PC implementations, especially at smaller CROs, as these do not always have Python on the PCs, and do now want to (or cannot) install these. It also makes it also slightly easier for integration in existing software such as our SDTM-ETL software, as depending on the customer, one can package the software with either the Windows, Linux or Mac executables, and the receiver can run the software "out of the box", without needing to have Python installation. SDTM-ETL will however both support both versions. For the Verisian version, it is then up to the customer to install both Python and the Verisian engine.
Another difference is that the Verisian engine supports datasets in SAS-XPT and in SAS7BDAT format, but not in the new Dataset-JSON format. Personally, I find that a pity, as it can be an argument for regulatory authorities such as the FDA to further delay to accept submissions in (modern) Dataset-JSON format. CDISC CORE engine accepts both XPT and Dataset-JSON.
Another difference is that the output Excel has a slightly different styling. Every implementer can however define their own styling as CORE allows to define their own report Excel template using the -rt (--report-template) parameter, providing a path to a custom Excel report template.
Regarding the Excel output: for the same SDTM test file, the CDISC engine may also report a number of "rule evaluation error - operation failed" issues. This can e.g. happen for a rule that checks against the define.xml but the define.xml itself has not been provided. Verisian does not report these as issues, but the rule is then simply skipped during execution. When a rule is skipped, the Verisian Excel also displays the reason why the rule was skipped in an additional column which is not present in the CDISC Excel. Personally, I like the Verisian approach better, as messages like "Rule evaluation error" or "Rule execution error" may be confusing and lead to panic by the users, having them think something is wrong with one of their datasets. Maybe CDISC could add an additional CLI parameter to skip such messages in the Excel report.
For real impact, it is now important that as many possible stakeholders, especially sponsors, large and small CROs as well as service providers and software companies start using CORE v.1.0.0 and share their
experiences with both Verisian and CDISC. Especially for Verisian it is important to get feedback on the validation of real and (very) large studies.
And, in case you find a bug or other software issue, please report them to
the project side at CDISC and/or the project side at Verisian.
Conclusion: CDISC CORE v.1.0.0 is a milestone. It is completely open source, including all the rules themselves and how they have been implemented (as human-readable YAML/JSON code), and found to be very robust. This means we can finally forget about
a single vendor having a propriety implementation that is completely "black box", and is notoriously known for its "false positives" and with a license that is at least doubtful on whether one can use it for production
work without needing to pay a huge amount of license fees. CORE also has this great feature that one can use or add own "custom rules", and group them into "custom standards" and add them to the engine
itself. This can be based on customer (for CROs), project, therapeutic area (for sponsors). Personally, I am of the opinion that each CRO and sponsor should start to develop such "custom rules" as CORE right now
(we will be happy to help).
CORE-1.0.0 comes in two different "flavors", one delivered by CDISC, and one by Verisian. This is fine as both use exactly the same ruleset and deliver the same results. The Verisian implementation
is probably more suited for use on servers and for very large datasets, but does not support Dataset-JSON, whereas the CDISC version is probably more suited for single PC usage at smaller organizations as, due to the Windows
executables, it can be run "out of the box" without anything else to be installed. The CDISC implementation may also be slightly easier to integrate in existing software. Our own SDTM-ETL mapping software will however
support both.
IMPORTANT: All in this article is my personal opinion!



