Prism 10 introduced an all new file format that unlocks an enormous amount of potential as to what can be done with Prism files - both inside and outside of the application - while making sure that the transition from using one of Prism's older file formats is nearly seamless. This page covers the philosophy and motivation behind the decision to introduce our new file format, and also provides some technical details on how the new file format is organized and how - if you choose - you can explore it!
Philosophy and motivation behind the new file format
More Open, More Accessible
Prism files contain an enormous amount of useful information about your work: from your raw data, to the analyses options that you've selected and the subsequent results, to the customized graphs that you create. But ultimately, this is all your work. And we believe that you should always have complete access to it. This was one of the primary motivations for us to introduce the new .prism file format as a means to make it easier to access these assets within the file itself.
Fundamentally, what has changed with the new Prism file format? First and foremost, the contents of your file are no longer hidden or obscured behind a proprietary format. Instead, all of the content inside the new Prism file format is stored within structures that can be fully explored and exampled (the new Prism file operates much like a zip file). Moreover, we chose to use industry-standard methods for storing each component of the file: data are stored in CSV format, graph preview images are stored in PNG format, and all analysis parameters and results are stored in standard JSON schemas. Ultimately this means that nothing gets locked away inside a file that can only be accessed if you're using Prism. On the contrary: our new file format makes it possible to explore the file contents both manually (human-readable) and programmatically (machine-readable) using third-party tools.
Transparency is key
Validation of scientific results is a critical component of any research project. This often means re-creating or repeating the work that was performed, using the same experimental conditions, the same data collection methods, and the same analysis procedures to reproduce the same results. Unfortunately, the inability to reproduce results - especially in biological, biochemical, and biomedical research - has become a common problem known as the "Reproducibility crisis". Importantly, this problem often has nothing to do with whether or not the results are "correct". Instead, the problem is due to the lack of transparency surrounding the methods used for data collection and analysis. When this information is unavailable or incomplete, even minor changes to the experimental process can lead to dramatically different (and thus, non-reproducible) results. This is another reason that we've chosen to transition Prism to a more open file format: every selected option for every analysis is readily available alongside every parameter and calculated analysis result. Your Prism file will provide a clear lineage of your analysis work all the way from input data to final results and visualizations.
Embracing FAIR ideologies
In 2016, an article titled "The FAIR Guiding Principles for scientific data management and stewardship" was published in Nature's Scientific Data journal and introduced the concept of FAIR data principles. The objective of these principles was to increase the Findability, Accessibility, Interoperability, and Reusability of scientific data which would - in turn - help improve the reproducibility of any work performed with the data. As part of the transition to our new file format, we have embraced much of the FAIR ideology. Any component of a Prism file (data, analysis parameter or result, or graph) can be located within our file structure, and is available for re-use in any desired subsequent manner. By using industry-standard formats (CSV, PNG, JSON, etc.), we ensure that these assets can be utilized even outside of Prism.
Whether you've just started using Prism or have been using it for decades, we know how important it is to retain access to your results (after all, this is why we chose to move to a new file format in the first place!). So we definitely didn't want you to lose access to any Prism files that you generated in one of our older file formats (with the .pzfx or .pzf file extension). But we also wanted to make sure that introducing a new file format would be as seamless as possible.
That's why we introduced "Compatibility Mode" with Prism 10. Essentially Prism 10 will allow you to open files in older formats in Compatibility Mode without warnings or alerts. Opening in Compatibility Mode won't limit you: you'll still have access to everything Prism 10 has to offer. Importantly, when you choose to save the file, Prism will automatically check to ensure that the file is still compatible with its existing file format. If there are no compatibility issues, Prism will simply save the file as you would expect. If any new functionality or features that are incompatible with the old file format are present, Prism will list each of these issues and provide you with options on how to proceed (including saving the file in the new .prism file format). These changes ensure that you can seamlessly transition to using our new file format without any worry about what happens to all of your existing work.
Some basic technical details on the new file format
The new Prism file format using the ".prism" file extension is actually a collection of many separate folders and files, compressed and archived together like a zip file. The individual files that make up the collection of objects within the .prism file represent different components of your overall project: such as your raw data, your graphs, your analysis parameters, analysis results, etc. In addition, the .prism file contains metadata which can be used (either manually or programatically) to find the relevant parts of the project.
To explore a .prism file manually, you can start by changing the file extension from .prism to .zip (warning: before proceeding, you may want to create a copy of your file just in case you accidentally change or modify some component of the file).
Once you've changed the file extension to .zip, you should be able to un-archive the contents of the file by double-clicking on it. Note: this typically creates a new folder with all of the contents of the file inside. Open that folder to view these contents. For the examples below, you may also choose to use the example file below.
When exploring the contents of a file in the new file format, it's recommended to begin with the "document.json" file. This will be at the top-level of the contents, and serves as a starting point for finding any other component of the file. Importantly, the "sheets" section lists the unique identifier (UID) for every sheet in the Prism project file, grouped by sheet type (data, analysis results, graphs, layouts, and project info). The UIDs look complex at first, but are just a means of ensuring that every single object within a Prism file can be uniquely identified with no duplicate UIDs for any two objects within the same file. The titles for each of the sheets in a Prism file are given in the "sheetAttributesMap" section, using the same UIDs as in the "sheets" section.
Find the data
Data from a data table in Prism is stored as a .csv file. To find this csv file, start by identifying the sheet UID (using the "sheetAttributesMap" and "sheets" section of the document.json file). In this example, the UID is D2BA2BA5-E5D3-4123-BEB0-EE8E83A725B9. Once you've located the UID, open the "data" folder, then the "sheets" folder. Inside, find the folder with the corresponding UID and open it.
This folder will contain a file called "sheet.json". Within this file, there is a section called "table" that includes another UID. In this example, the table UID is 41482C82-CAB8-4E5C-AA4E-0F3AECC50B71. With this new UID, go back to the "data" folder and open the "tables" folder. Inside, find the folder with the corresponding UID and open it. This folder will contain a file called "data.csv" that contains the raw values stored in the Prism data table.
Find the graph
To find a specific graph, start by identifying the sheet UID (using the "sheetAttributesMap" and "sheets" section of the document.json file). In this example, the UID is C0350155-E0F6-407F-9DB9-FDE70CA6BC29. Once you've located the UID, open the "graphs" folder and find the folder inside with the corresponding UID (in this case, there's only one graph sheet, so this is simple).
Inside this folder, a file called "preview.png" can be found. This is an image file of your graph as you've customized it. Note that only graphs that you've viewed in the application itself will have this "preview.png" file (Prism does not create previews for graphs that haven't been viewed in the application).
Find the analysis results
There are two ways that Prism stores analysis results:
1.The table "view" that you would recognize as the results sheet
2.The complete "raw" results that the analysis generates to populate the results sheet
Finding either one of these can be done in a similar fashion. Start by identifying the sheet UID (using the "sheetAttributesMap" and "sheets" section of the document.json file). In this example the nonlinear regression analysis UID is 0D9656DF-9344-4539-B718-EC3D7BFEC43E. Once you've located the UID, open the "analyses" folder and find the folder inside with the corresponding UID (in this case, there's only one analysis sheet, so this is simple).
Inside this folder, a file called "results.json" can be found. This is the "raw" results of the analysis, with values stored with as many decimals as possible within JSON schema's specific to each analysis.
To find the table "view" as it's presented in the Prism UI, it requires a few more steps. First, open the file "sheet.json". The section "resultSheets" lists each tab (both visible and hidden) of the results along with their own UID. For this example, we want to see the "Table of results", so we'll want to use the UID 1CC9F339-0217-4AD5-92EC-C610EE86ECAC. Once you've located the UID, open the "result_sheets" folder and find the file inside with the corresponding UID.
This file will have a section called "dataSheet" with another UID. In this example, the UID is 9B99432B-6CEC-4C91-A7A0-C167DFC6C466. Using this UID, go all the way back to the root (top level) of the folders (where document.json was located) and find the "data" folder. Open it, then open the "sheets" folder. Inside, find the folder with the corresponding UID and open it.
This folder will contain a file called "sheet.json". Within this file, there is a section called "table" that includes another UID. In this example, the table UID is D1815795-1D83-4A99-B5FB-7B4B32467F70. With this new UID, go back to the "data" folder and open the "tables" folder. Inside, find the folder with the corresponding UID and open it. This folder will contain a file called "data.csv" that contains the results as they're presented in the table view within Prism.
