ready4 – Project

Docs: Motivation

Mon, 01 Jan 0001 00:00:00 +0000

Problem

Health economic models are a potentially useful tool to help health decision makers navigate complexity, but can have significant limitations such as:

Mistakes: Errors, common in even relatively simple health economic models, become both more likely to occur and more difficult to detect as model complexity grows;
Poor transparency: the validity of model analyses can be difficult to adequately ascertain if it is not clear who contributed to the model, what assumptions they made, how model algorithms were implemented, how those algorithms were tested and what data they were applied to;
Contested legitimacy: the value judgments of the model development team (e.g. what types of question are most important for a model to address, what parts of the workings of the system of interest to represent and in what detail, what outcome variables to include, which stakeholders to consult, etc) may differ from those using or affected by model outputs;
Narrow applicability: a model might be too simple to adequately explore some problems and too complex to reliably address others and be hard to transfer beyond a very specific decision context (e.g. within a specific jurisdiction);
Limited interoperability: different approaches to model implementation, dissemination, ownership and reporting makes it more difficult for multiple models to be efficiently and safely combined;
Ease of misuse: in the absence of clear documentation and prominent caveats, a model can easily be applied to decision problems to which it is poorly suited (potentially supporting decisions with serious negative consequences);
Restricted access: a potential overcompensation for fear of model misuse is constructing high barriers to accessing model code and data - thus limiting model testing, reuse and refinement; and
Growing stale: health economic models are rarely updated, meaning they can lose validity with time (e.g. input data becomes less relevant, new better performing algorithms are not incorporated, sudden major changes in environment / epidemiology / policy / service system are not accounted for).

Reponse

To help address these issues, the ready4 software framework aims to support transparent, reusable and updatable model implementations using R.

Docs: Our priorities

Thu, 05 Jan 2017 00:00:00 +0000

Docs: Contribution types

Thu, 05 Jan 2017 00:00:00 +0000

Docs: Overview

Mon, 01 Jan 0001 00:00:00 +0000

What is ready4?

ready4 is a prototype software framework for developing living and transferable health economic models and applying those models in reproducible and replicable research. The project is implemented in the open source language R and is led by researchers at Monash University.

What is a software framework?

A software framework is a shared technology that can be used by multiple teams to collaboratively author software.

What is a computational model?

A computational model is a simplified representation of a system of interest that is implemented in computer code.

What makes a ready4 model modular?

The paradigms we use for computational model development allow individual model components (modules) to be independently reused (in other models) and safely and flexibly combined (to model more extensive systems).

Why is that useful?

We hope that ready4 will help address significant limitations in how health economic models are implemented, encourage data sharing and facilitate modelling analyses that generate valuable insights for health policymakers and system planners.

Who is it for?

ready4 is designed to be primarily used by coders and modellers working collaboratively on computational modelling projects in which other stakeholders such as funders and researchers make essential contributions.

What is it being used for?

Currently, ready4 is being applied to develop and apply a modular model of youth mental health.

Can I use it?

ready4 is publicly available and free for you to assess (to verify and validate), apply (to use and/or author health economic model modules) and to derive your own derivative works from (to leverage and enhance the work of others) under liberal terms of use.

Why is it a prototype?

Currently, most of the ready4 software suite is available in the form of development releases. That means the libraries implementing ready4 may require more development, documentation and testing before it could be confidently used for scientific purposes other than the specific studies to which the ready4 development team have already applied it.

Why is it called ready4?

ready4 is short-hand for “readyforwhatsnext”, the title of the research project that catalysed the framework and a reference to how computational models can improve the preparedness of decision makers and system planners.

Where should I go next?

We’d recommend reading the documentation in the order in which sections appear in the table of contents (so go next to Examples, then to Getting started and so on). Scientific manuscripts on the framework’s ethical underpinnings and early implementation are also available.

Docs: Before you get started

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Model

Mon, 01 Jan 0001 00:00:00 +0000

A model is a simplified representation of a system of interest. In the way we use the term, we also mean that a model is:

abstract and general (i.e. largely free of non-modifiable data, including numeric values, that are assumption- or context- specific) and
a tool (i.e. a model can be used to help undertake an analysis, it is not the analysis itself).

If a model is used to help solve economic problems (e.g. those arising from scarcity) relating to health and healthcare it is a health economic model. Many health economic models are developed to inform a decision or set of decisions (e.g. relating to youth mental health policy and system design), in which case they can also be called a decision model.

Ideally, a health economic model should have three inter-related representations - conceptual, mathematical and computational.

Conceptual Model

A conceptual model refers to underlying theory and beliefs about a system of interest that can be described in words and pictures.

Mathematical Model

A mathematical model formalises a conceptual model as a set of equations.

Computational Model

A computational model implements the conceptual and mathematical models of a system of interest as computer code.

Computational models can take a modular approach to implementation.

Docs: Module

Mon, 01 Jan 0001 00:00:00 +0000

A modular computational model is one that constructed from multiple self-contained components, called modules. The advantages of developing a modular model include:

each module can be independently re-used in other computational models;
a complex model can be developed iteratively, beginning with a simple representation that is easier to validate and then progressively expanding in scope and functionality through the addition of new modules, to be validated independently and jointly; and
long term and resource intensive model development is more feasible through combining multiple independently managed and financed modelling projects.

To ensure that all ready4 computational model modules can be safely and flexibly combined, each module is created from a template using authoring tools that support standardisation.

Docs: Coders

Mon, 01 Jan 0001 00:00:00 +0000

If you are a coder, you may be a data scientist or software engineer by training or are perhaps a quantitative researcher who spends a high proportion of their time writing code to undertake their work.

Role

The primary role of coders in ready4 modelling projects is to author modules that implement computational models.

Tools

The ready4 tools of most use to coders are the software framework libraries for authoring modules.

Benefits

ready4 provides an opportunity to write software that matters! Our aim is to help improve population health through empowering decision makers with better models. If you already write code for health-economic modelling projects, the ready4 software framework may help you enhance your impact (facilitating code re-use) and work-efficiency (through partial automation of code development and quality-assurance workflows).

Contributing to ready4

The types of contribution you can make to ready4 include:

Docs: Modellers

Mon, 01 Jan 0001 00:00:00 +0000

If you are a modeller, you are responsible for the overall implementation of a modelling study from initial conceptualisation through to analysis and reporting. You are likely to be an economist, epidemiologist or statistician and are probably reasonably comfortable with writing analysis scripts in statistical software (potentially including R), without necessarily being a coding wizard.

Role

The primary role of modellers in ready4 modelling projects is to use modules to undertake analyse as part of modelling projects.

Tools

The ready4 tools of most use to modellers are the software framework libraries for authoring model datasets and analyses and model module libraries for use in computational modelling.

Benefits of using ready4

We hope that ready4 can be of benefit to you by helping you to efficiently build on work by other modellers, to implement more reproducible workflows, and to share your work so that it can be reused.

Contributing to ready4

The types of contribution you can make to ready4 include:

Docs: Modelling project

Mon, 01 Jan 0001 00:00:00 +0000

A ready4 modelling project will use the ready4 software framework to implement the three steps of:

Developing and validating a computational model;
Adding context-specific data to that computational model; and
Applying the computational model to the supplied data to undertake analyses.

The key components of each step are summarised here.

Docs: Planners

Mon, 01 Jan 0001 00:00:00 +0000

If you are an planner, you contribute to policy development or service planning that aims to improve population health. You probably value the role of modelling to inform your work, but are likely to rely on others to provide much of the technical expertise to implement computational models.

Role

The primary role of planners in ready4 modelling projects is co-design and use of models to support decision making.

Tools

The ready4 tools of most use to planners are user-interfaces that convert computational models into useful decision aids.

Benefits of using ready4

We hope that ready4 can provide you with transparent reusable and updatable decision support.

Contributing to ready4

The types of contribution you can make to ready4 include:

Docs: Repeatable research

Mon, 01 Jan 0001 00:00:00 +0000

Although there is widespread support from the scientific community on the importance of repeatable research, the definition of key terms such as reproducibility and replicability can vary across disciplines and methodologies (e.g. the extent to which computational modelling is used). The meanings we intend when using these terms are described below.

Reproduction and Replication

The distinctions we make between reproduction and replication have been guided by the approach outlined in a report by the National Academies of Sciences, Engineering and Medicine. However, we have adapted their definitions slightly as the meanings in that report were framed in terms of study findings / outcomes, whereas our usage relates more to intended objectives when deploying tools.

Our usage

Reproduction

Applying the same analysis code to the same input data with the expectation of generating identical outputs (with the exception of trivial artefacts like datestamps for when analysis reports were produced).

Replication

Applying analysis code used in a study to new input data. The analysis code is reused with only minimal edits that are necessary to account for differences in input data paths and variable names and to study metadata (e.g. investigator names, sample descriptions). The new data can be real or fake, but will include the same structure and concepts / measures as those found in the original study’s dataset. If the new data is a sample from the same population as the original study, then the expectation when undertaking replications is for results across studies to be broadly similar.

Suggested policies

When publishing analysis code we suggest adopting the following rules of thumb:

If the data required to re-run a study analysis are publicly available (or declared by the analysis program itself), publish the code as a reproduction program (e.g. this program for creating a synthetic population).
If the data required to re-run a study analysis are not publicly available, we publish the replication version of the code. The replication version of the code may be configured to ingest a synthetic (fake) representation of the study dataset as with this utility mapping replication program. Details of the (minimal) steps required to revert the replication code to a version that can be used for reproduction purposes are typically embedded within the program itself.

Docs: Users

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Community members

Mon, 01 Jan 0001 00:00:00 +0000

Community members have an important role to play in both the development of the overarching ready4 model and the applicability of ready4 to specific decision problems.

One of the main contributions that community members can make is to provide advice. To date, the advice we have elicited from community members has related to shaping the design and conduct of individual modelling projects. In the medium term, he hope to supplement these opportunities through growing a ready4 support community.

Docs: Funders

Mon, 01 Jan 0001 00:00:00 +0000

There are six main types of funder that can provide cash and/or in kind support to ready4:

Grant making research bodies can support modelling project proposals submitted to their existing funding schemes. These types of funder could also consider making a number of changes to how they work including the assessment weightings and levels of financial support given to the reproducibility, replicability and transferability components of research proposals and initiating targeted calls for proposals to improve the transparency, reuse and maintenance of models to inform policy.
Government departments can support the development of ready4 as part of programs to enhance data analysis and modelling capability in youth mental health by providing support to develop core ready4 infrastructure (e.g. our software maintenance and community development priorities). When commissioning new modelling projects, governments could make providing open access to code and (to the greatest extent feasible, balancing confidentiality considerations) data a requirement of all applicants.
Youth mental health service commissioners can commission data analysis and modelling projects that develop novel decision aids and to apply existing ready4 modules to undertake new analyses.
Philanthropic donors can help accelerate our development and enhance our impact by supporting us to bring our existing in-development software to launch and to further extend the ready4 model.
Corporate sponsors can provide cash, expertise and free product licenses to support both our core open-source infrastructure and individual modelling projects.

Docs: Researchers

Mon, 01 Jan 0001 00:00:00 +0000

Researchers in multiple discipline enhance prior, current or planned future projects related to how environmental, social and technical systems shape health-economic outcomes by using ready4 to:

label and share non-confidential data (e.g. tables of summary statistics / parameter values) from prior and current work to extend access and impact;
working with modellers to undertake modelling analyses;
working with coders to redevelop study analysis scripts by authoring modules that are transferable to other contexts;
supervising student projects to assist with ready4’s current priorities; and/or
develop research proposals that incorporate a modelling project.

Researchers considering using ready4 should ensure they understand the development status of the tools they wish to use. If the required software is not yet a production release (a process we are working on!) we’d suggest only using it for testing or exploratory work that is not designed to inform decision making. All our software is free and open source so you don’t need to ask our permission to use it. We are however, very happy to discuss ideas for potential collaborations - contact the project lead to arrage a chat.

We also welcome advice from researchers about how we can make ready4 more relevant and useful.

Docs: Stakeholders

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Transferability

Mon, 01 Jan 0001 00:00:00 +0000

It is common for discussions of scientific studies to consider the extent to which findings can be generalised (e.g. if a well conducted study concludes with high confidence that an intervention is cost-effective in Australia, is it valid to infer that it is likely to be cost-effective in the United Kingdom?). However, we are more interested in the transferability of computational models (e.g. the extent to which the data-structures and algorithms from a computational model developed for an Australian context can be used to explore similar topics in the United Kingdom). Our usage of the term “transferring” (and by extension “transferability”, “transferable”, “transfers”) reflects this motivation.

Transferring - our meaning

Adapting a computational model, in whole or in part, to extend the types of data and/or research questions to which it can be applied. The new types of data will possess some differences in structure and / or concepts from that to which the computational model had previously been applied and these differences may be why research questions need to be reformulated.

When we use the term transferring, we are typically referring to either (a) authoring or (b) using on of the following:

An analysis program (or sub-routine) that has been adapted from an executable from another study to account for differences in the input data / research question.
Inheriting data-structures and algorithms that selectively re-use, discard and replace elements of a study’s computational model based on an alternative use-case.
(Multi-purpose) function libraries that have been created by decomposing a study’s (single purpose) analysis program.

Examples

The scorz module library was originally developed to provide an R implementation of algorithms in other languages for scoring adolescent AQoL-6D health utility as part of a utility mapping study (which also used the analysis program mentioned above). Examples of all three approaches mentioned in the previous section can be seen by examining the documentation and source code of the scorz library:

Two vignette programs from the scorz library website score different utility instruments. The first program scores adolescent AQoL-6D health utility and acts as a template for the second, which has been modified to score EQ-5D health utility.
Inspecting those example programs shows that one of the key adaptations in the EQ-5D program is to use the ScorzEuroQol5 module instead of the ScorzAqol6Adol module. Both of these modules inherit from ScorzProfile. This arrangement means that all three modules share some features (in terms of both structure and algorithms) but selectively differ (e.g. aspects that are necessarily different for scoring different instruments).
The algorithms attached to each module from the scorz library are principally implemented by functions (the source code for which can be viewed here) that were created when decomposing an early draft of the above mentioned study algorithm. These functions are called by module methods (source code viewable here).

Docs: Provide advice

Thu, 05 Jan 2017 00:00:00 +0000

What?

We need advice:

to help review and update our priority goals and develop, refine and implement strategies for achieving these goals;
to help plan and execute modelling projects that produce transparent, reusable and updatable computational models; and
to identify how our existing software and data can be usefully improved.

Who?

We wan advice from our users (coders, modellers and planners), stakeholders (funders, researchers and community members) and other supporters (those with relevant expertise in technical communication, building open source communities, product development, etc).

How?

Advice can be provided by:

Participate in the advisory structures and events of individual modelling projects. The nature of these opportunities will vary by project and the team responsible for implementing each project.
Flag software features, usability and documentation issues. If you have the capacity and willingness to also fix the issues you can approach this using the process for making a code contribution. Otherwise, you can do so by creating an issue on that software projects repository in our GitHub organisation. For example, to create a new issue relating to the ready4 foundation library, use https://github.com/ready4-dev/ready4/issues/new (you will need a GitHub account).

Docs: Contribute code

Mon, 01 Jan 0001 00:00:00 +0000

What?

Test, improve or extend our software. This is essential to us achieving our following priority goals:

Priority 1. Launching the ready4 MVP systems model.
Priority 2. Maintaining ready4.
Priority 3. Applying ready4.
Priority 4. Growing a ready4 community.

Who?

To make a code contribution, you need to be a coder familiar with R, R Markdown and git. You will also need a GitHub account. For many types of contribution, you will also need to use our framework’s module authoring tools. We have yet to adequately document and refine these tools to make them easier for third parties to use (we plan to do this), so if you are interested in making anything other than a relatively minor code edit, we recommend that you first contact our project lead to discuss your idea.

As a contributor to ready4, you will also be expected to adhere to the

How ?

The process for making a code contribution, broadly conforms to the steps we itemise below, that we have minimally adapted from this template. If you need further help to make a contribution, you can contact the ready4 project lead directly.

Find an issue that you are interested in addressing or a feature that you would like to add. Ideally consider how your planned contribution matches our current priorities.
Fork the repository associated with the issue from our GitHub organization to your local GitHub organization. This means that you will have a copy of the repository under your-GitHub-username/repository-name.
Clone the repository to your local machine using:

git clone https://github.com/github-username/repository-name.git

Create a new branch for your fix using:

git checkout -b branch-name-here

Make the appropriate changes for the issue you are trying to address or the feature that you want to add.
To add the file contents of the changed files to the “snapshot” git uses to manage the state of the project, also known as the index, use:

git add insert-paths-of-changed-files-here

To store the contents of the index with a descriptive message, use:

git commit -m "Insert a short message of the changes made here"

Push the changes to the remote repository using:

git push origin branch-name-here

Submit a pull request to the upstream repository.
Title the pull request with a short description of the changes made and the issue or bug number associated with your change. For example, you can title an issue like so “Added more log outputting to resolve #4352”.
In the description of the pull request, explain the changes that you made, any issues you think exist with the pull request you made, and any questions you have for the maintainer. It’s OK if your pull request is not perfect (no pull request is), the reviewer will be able to help you fix any problems and improve it!
Wait for the pull request to be reviewed by a maintainer.
Make changes to the pull request if the reviewing maintainer recommends them.
Celebrate your success after your pull request is merged!

Docs: Contributions

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Fund projects

Thu, 05 Jan 2017 00:00:00 +0000

What?

Provide cash or in-kind resources to support us to achieve any or all of our priority goals:

Priority 1. Launching the ready4 MVP systems model.
Priority 2. Maintaining ready4.
Priority 3. Applying ready4.
Priority 4. Growing a ready4 community.
Priority 5. Extending ready4.
Priority 6. Integrating ready4 with other tools.

Who?

We are seeking support from multiple different types of funder. At this early stage of our development we would expect that the most impactful way of supporting ready4’s development will be to award funding for that purpose directly to ready4’s two institutional sponsors: Orygen and Monash University. Other ways to support ready4 will be to fund ready4 modelling projects led by other research institutions and which may or may not be formally affiliated with us.

How?

The two main categories of funding we seek are:

Core infrastructure. Essential to the success of priorities 1-2 and 3-6 above is adequately resourced support infrastructure. Financial support we receive for this purpose will primarily be dedicated to recruit a skilled team of data scientists (coders), modellers, technical documentation / training developers, community builders and stakeholder managers. Other important resource requirements relate to licensing appropriate technical solutions (hosting, security, workflow optimisation, etc) to support the ready4 community.
Modelling projects To advance priorities 3 and 5 above, teams with high quality plans to undertake modelling projects with ready4 need to be backed with financing. Typically funding provided to these types of projects will be primarily spent on employing modellers, data-scientists and other researchers and on supporting processes to meaningfully engage community members, planners and other stakeholders.

If you would like to invite a funding proposal from ready4, contact the project lead. You can also simply make a direct donation to Orygen (please remember to specify www.ready4-dev.com as the reference for the project you would like to support!).

Docs: Undertake projects

Thu, 05 Jan 2017 00:00:00 +0000

What?

A ready4 modelling project undertakes novel analysis of youth mental health topics by using, enhancing and/or authoring model modules, datasets and executables. Each ready4 modelling project has its own unique funder(s), governance, objectives and team. The links between modelling projects are in the form of a common framework and membership of a collaborative community.

Undertaking modelling projects will help us achieve our following priority goals:

Priority 3. Applying ready4.
Priority 5. Extending ready4.

Who?

Modelling projects should typically be led by a researcher (who may or may not be a modeller) or planner. The core project team will always include modelling expertise and, should authorship of new modules (or extensions to existing modules) be required, will also need to include coders. Advisory structures to engage community members and planners are also recommended.

How?

There are three main steps in implementing a ready4 modelling project.

Step 1: Develop model

Each project’s computational model is constructed by adopting one or more of the following strategies:

selecting a subset of existing ready4 modules and using them in unmodified form;
selecting a subset of existing ready4 modules and contributing code edits to these modules to add desired functionality;
selecting a subset of existing ready4 modules and using them as templates from which to author new inheriting modules (which can be code contributions to an existing module library or distributed as part of a new library); and/or
authoring new ready4 modules (most likely to be distributed in new code libraries).

As part of the validation and verification process for all new and derived modules, tests should be defined, bundled as part of the relevant module libraries and rerun every time these libraries are edited.

Step 2: Add data

By data we typically mean digitally stored information, principally relating to model parameter values, that can be added to the ready4 computational model to tailor it to a specific decision context (e.g. a particular population / jurisdiction / service / intervention) and set of underpinning beliefs (e.g. preferred evidence sources). Data for a ready4 modelling project will be from one or both of the following options:

finding and using existing open access data from other ready4 projects;
supplying new project specific data, appropriately describing these data and (for non-confidential records) sharing these data publicly.

Step 3: Run analyses

ready4 project analyses apply algorithms contained in ready4 modules to supplied data to generate insight and can be implemented by:

adapting existing replication programs;
authoring new analysis programs; and / or
developing a user-interface to allow non-technical users to run custom analyses.

When reporting analyses, using a reporting template can be useful.

Docs: Support the ready4 community

Thu, 05 Jan 2017 00:00:00 +0000

What?

Help other members of the ready4 community to apply ready4 by authoring documentation, developing training and posting answers in online help. This support is essential for us to advance the following project goals:

Priority 2. Maintaining ready4.
Priority 4. Growing a ready4 community.
Priority 5. Extending ready4.

Who?

Any community member (user or other stakeholders) can help us to improve the accessibility, clarity and usefulness of our documentation. Coders and modellers are particularly welcome to contribute support that leverages their technical expertise.

How?

The types of support that we welcome contributions on include:

Improving the documentation contained on this website. To do this, you will need a GitHub account. Once you have that, you can:

flag a general issue and suggest improvements by clicking on the “Create documentation issue” link or visiting https://github.com/ready4-dev/ready4web/labels/documentation ; and/or
suggest edits to a specific page by clicking on the “Edit this page” link.

Improve the documentation for specific library, executable or dataset:

for software documentation edits, you can use the same workflow as that for making a code contribution; and
for improvements to dataset documentation, we have yet to set up a streamlined workflow for this process, so for moment please contact the ready4 project lead directly if you ar interested in making this type of contribution.

Contributing to developing other training and support resources (e.g. answering questions in online help, video turorials, etc). We believe that this type of content is most likely to become relevant when we have made more progress in developing the ready4 community. But again, if you are interested in this area, please contact the project lead to discuss.

Docs: Contributor covenant (code of conduct)

Mon, 01 Jan 0001 00:00:00 +0000

Our pledge

We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.

Our standards

Examples of behavior that contributes to a positive environment for our community include:

Demonstrating empathy and kindness toward other people
Being respectful of differing opinions, viewpoints, and experiences
Giving and gracefully accepting constructive feedback
Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
Focusing on what is best not just for us as individuals, but for the overall community

Examples of unacceptable behavior include:

The use of sexualized language or imagery, and sexual attention or advances of any kind
Trolling, insulting or derogatory comments, and personal or political attacks
Public or private harassment
Publishing others’ private information, such as a physical or email address, without their explicit permission
Other conduct which could reasonably be considered inappropriate in a professional setting

Enforcement responsibilities

Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.

Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.

Scope

This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement. All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the reporter of any incident.

Enforcement guidelines

Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:

1. Correction

Community Impact: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.

Consequence: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.

2. Warning

Community Impact: A violation through a single incident or series of actions.

Consequence: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.

3. Temporary ban

Community Impact: A serious violation of community standards, including sustained inappropriate behavior.

Consequence: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.

4. Permanent ban

Community Impact: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.

Consequence: A permanent ban from any sort of public interaction within the community.

Attribution

This Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.

Community Impact Guidelines were inspired by Mozilla’s code of conduct enforcement ladder.

For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations.

Docs: Terms of use

Mon, 01 Jan 0001 00:00:00 +0000

Docs: Open source licensing

Mon, 01 Jan 0001 00:00:00 +0000

To help ensure the models we develop are as transparent as possible and to make their algorithms as useful to others as possible, all ready4 software is free and open-source. You are encouraged to make as widespread use of our software, including the creation of derivative works, as you see fit, so long as it is consistent with each item’s license. Our software is typically licensed under GPL-3, a copy-left open-source licensing regime.

Docs: Citing ready4

Mon, 01 Jan 0001 00:00:00 +0000

To make it easier to cite our software, each software item bundle includes a CITATION.cff file. Inclusion of this file means that the repositories storing our software can generate appropriate citations in the format of most relevance to you.

Currently:

Zenodo provides a free text field under the heading “Cite as” which enables you to generate a wide range of citation manager and journal specific citation outputs. There is also an “Export” tool that will generate citation metadata in multiple output formats;
OpenAire Explore has a “Cite this software” button that allows you to generate a citation in multiple journal formats or to download BibTeX or RIS files;
Github repositories have a “Cite this repository” button that can generate both BibTeX and APA output as well as link to the Citation.cff file.

Additionally, we have included a CITATION file in each of our R libraries so that you can generate a citation from within an R session using the citation function (for example: citation("ready4").

Docs: Disclaimer

Mon, 01 Jan 0001 00:00:00 +0000

All ready4 software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Furthermore, no ready4 software is yet sufficiently well documented and tested to be given a production release. All ready4 software should therefore viewed as experimental development releases.

Docs: Priority 1: Launch ready4 Minimum Viable Product (MVP)

Thu, 05 Jan 2017 00:00:00 +0000

Why?

All our software, regardless of status is supplied without any warranty. However, our views about whether an item of software is potentially appropriate for others to use in undertaking real world analyses can be inferred from its release status. If it is not a production release, we probably believe that it needs more development and testing and better documentation before it can be used for any purpose other than the specific studies in which we have already applied it. Partly for this reason, it is unlikely that any item of our software will be widely adopted until it is available as a production release. We also cannot meaningfully track uptake of our software until it becomes available in a dedicated production release repository. We need a critical mass of modules available as production releases so that they can be combined to model moderately complex systems.

What?

By bringing all our current development version and pipeline libraries to production release, we aim to launch the ready4 Minimum Viable Product (MVP). The MVP will comprise our software framework plus an initial skeleton of production ready modules for modelling people, places, platforms and programs.

The most important types of help we need with achieving this goal are funding, code contributions, community support and advice.

How?

The main tasks to be completed to bring all of our existing code libraries to production releases is as follows:

(For unreleased software) Address all issues preventing public release of code repositories (e.g. fixing errors preventing core functions working, removing all traces of potentially confidential artefacts from all versions/branches of repository, etc.).
(For code libraries are implemented using only the functional programming paradigm) Author and test new modules.
Write / update unit tests (tests of individual functions / modules for multiple potential uses / inputs that will be automatically run every time a new version of a library is pushed to the main branch of its development release repository).
Enhance the documentation that is automatically authored by algorithms from our model authoring tools. This will involve some or all of:

minor modifications of function names / arguments / code;
updating the taxonomies datasets used in the documentation writing algorithm; and/or
updating the documentation authoring algorithms (within the ready4fun and ready4class packages).

Adding human authored documentation for the modules contained in each library.
(For some libraries) Adding a user-interface.

When?

Our production releases will be submitted to the Comprehensive R Archive Network (CRAN). CRAN does not allow for submitted R packages to depend on development version R packages, so the dependency network of our code-libraries shapes the sequence in which we bring them to production releases.

The planned sequence for bringing current development release code libraries to production releases is:

Three of the six framework libraries - the ready4 foundation followed by the ready4show and ready4use authoring tools.
The nine module libraries that are sufficiently developed to have been used in real world scientific studies or policy analyses, in the following order: youthvars, scorz, specific, TTU, youthu, mychoice, vicinity, aus and heterodox.
The very early stage bimp library from the modelling programs pipeline,
The three remaining computational model authoring tools framework libraries, starting with ready4fun, then ready4class and finally ready4pack.

The planned sequence for bringing unreleased code first to development releases and then to production releases is:

The two development module libraries from the modelling places pipeline (the prevalence predictor and user interface libraries).
The two module libraries from the modelling people pipeline (the synthetic household creation library and that agent based modelling library).
The three module libraries from the modelling platforms pipeline (the primary mental health service discrete event simulation, followed by the early psychosis cohort model, followed by the service system eligibility and referral policy optimisation model).

How quickly we can launch production releases of all our code depends on how much and what type of help we get. Working within our current resources we expect the first of the 23 libraries listed to be released the first half of 2024 and the last during late 2026. With your help this release schedule can be sped up.

Docs: Priority 2: Maintain ready4

Thu, 05 Jan 2017 00:00:00 +0000

Why?

A significant limitation of many health economic models is that they are not updated and can become progressively less valid with time. The importance of maintaining a computational model increases if, like ready4, it is intended to have multiple applications and users. As we progressively make production releases to launch theready4 MVP model, we intend that people will start using it. As ready4 becomes more widely used, its limitations (errors, bugs, restrictive functionality and confusing / inadequate documentation) are more likely to become exposed and to require remediation. Addressing such issues needs to implemented skillfully and considerately to avoid unintended consequences on existing model users (e.g. to ensure software edits to fix one problem do not prevent previously written replication code or downstream dependencies from executing correctly). Open source projects like ready4 also need to make changes in response to decisions by third parties - such as edits to upstream dependencies and changes in the policies of hosting repositories and to update citation / acknowledgement information to appropriately reflect new contributors.

What?

All ready4 software needs to be maintained and updated to identify and fix bugs, enhance functionality and usability, respond to changes in upstream dependencies and to conscientiously deprecate outdated code. Open access datasets made available for use in modelling analyses need to be actively curated to ensure they remain relevant to current decision contexts. Decision aids need to be reviewed and updated to ensure they continue to use the most up to date and appropriate modules and input data.

The most important types of help we need with this priority area are funding, code contributions, community support and advice.

How?

The main tasks for the maintenance of framework and model software are to:

Appropriately configure and update the settings of the ready4 GitHub organisation and its constituent repositories to facilitate easy to follow and efficient maintenance workflows.
Proactively:

author ongoing improvements to software testing, documentation and functionality;
make archived releases of key development milestones in the ready4 Zenodo community; and
submit new production releases to the Comprehensive R Archive Network (CRAN).

Reactively elicit, review and address feedback and contributions from ready4 community (e.g. bugs, issues and feature-requests).

The main tasks for curating model data collections include:

Implementing ongoing improvements and updates to meta-data descriptors of data collections and individual files.
Facilitating the linking of datasets to and from the ready4 Dataverse.
Reviewing all collections within the ready4 Dataverse to identify datasets or files that are potentially out of date.
Creating and publishing new versions of affected datasets with the necessary additions, deletions and edits and updated metadata. Prior versions of data collections remain publicly available.
Informing the ready4 community of the updated collections.

The main tasks for curating decision aids include:

Monitoring the repositories of the software and the data used by the app for important updates.
Deploying an updated app bundle of software and data to a test environment on Shinyapps.io.
Testing the new deployment and elicit user feedback.
Implementing any required fixes identified during testing.
Deploying the updated app to a Shinyapps.io production environment.
Informing the ready4 community of the updated decision aid.

When?

Maintenance is an ongoing and current responsibility. Maintenance obligations are expected to grow considerably as we launch more production releases, extend the ready4 model and grow the ready4 community.

Docs: Priority 3: Apply ready4 to undertake replications and transfers

Thu, 05 Jan 2017 00:00:00 +0000

Why?

In this relatively early stage of ready4’s development, the authoring of new ready4 modules can involve a significant investment of time and skills, an investment that is typically made in the context of implementing a modelling project for a scientific study. However, once authored, these modules may significantly streamline the implementation of modelling analyses that replicate or transfer the studies for which they were developed. For modellers and other researchers, using ready4 for this purpose may provide the highest reward to effort ratio of any contribution to the ready4 community. Network effects also kick in - more replications and generalisations mean more open access data and module customisations available to other users, enhancing the practical utility of ready4.

What?

We plan to demonstrate that studies implemented with ready4 are relatively straightforward and efficient to replicate and transfer. The most important initial types of help we need with achieving this goal are funding, projects, code contributions and advice.

How?

The main tasks for implementing study replications and transfers are:

Identify the example study to be replicated or transferred.
Review that study’s analysis program:

do the data used in this program have similar structure / concepts / sampling to the data for which a new analysis is planned?
are modules used in that program from production release module libraries and do any of them require authoring of inheriting modules to selectively update aspects of module data-structures or algorithms?

Create a new input dataset, labelling and (for non-confidential data) storing the data in an online repository (which can be kept private for now).
(If new inheriting modules are required) Make a code contribution to create and test new inheriting modules.
Adapt the original study’s analysis program to account for differences in input data, model modules and study reporting.
Share new new analysis program in the ready4 Zenodo community.
Ensure the online model input dataset is made public and submit it as a Linked Dataverse Dataset in the appropriate section of the ready Dataverse.

When?

In most cases, we recommend waiting until production releases of relevant module libraries are available. However, we are currently planning or actively undertaking some initial study analysis transfers using the development versions of our utility mapping and choice modelling module libraries. We are undertaking this work in parallel with testing and, where necessary, extending the required modules. We suggest that, should you believe that any of our development version software is potentially relevant to a study you wish to undertake, you first get in touch with our project lead to discuss the pros / cons and timing of using this software.

Docs: Priority 4: Grow a ready4 community

Thu, 05 Jan 2017 00:00:00 +0000

Why?

ready4 is open source because we believe that transparent and collaborative approaches to model development are more likely to produce transparent, reusable and updatable models. No one modelling team has the resources or breadth of expertise and diversity of values to adequately address all of the important decision topics in youth mental health systems design and policy. Opportunities for modellers to test, re-use, update and combine each other’s work help make modelling projects more valid and tractable. Models have become increasingly complex, so simply publishing model code and data may have limited impact on improving model transparency. These aretefacts also need to be understood and tested. Clear documentation and frequent re-use in different contexts by multiple types of stakeholder make it more likely that errors and limitations can be exposed and remedied. Decentralising ownership of a model to an active community can help sustain the maintenance and extension of a model over the long term and mitigate risks and bottlenecks associated with dependency on a small number of team members.

What?

Our aim is to enhance the resilience, quality, legitimacy and impact of ready4 by developing a community of users and contributors. The most important initial types of help we need with achieving this goal are funding, community support and advice.

How?

The process of developing the ready4 community involves the following tasks:

Creating and recruiting to volunteer advisory structures to elicit guidance on strategic, technical and conceptual topics.
Enhancing the ease of use for third parties of existing framework authoring tools.
Developing improved documentation and collateral (e.g. video tutorials) for ready4 software and data.
Configuring hosting repositories to implement clear collaborative development workflows.
Promoting ready4 to potential users and stakeholders.
Continually expanding, diversifying and updating the authorship and maintenance responsibilities of all ready4 software.

When?

The speed at which we undertake activities to grow the ready4 community depends on our success at securing funding to provide required support infrastructure.

Docs: Priority 5: Extend the scope of the ready4 model

Thu, 05 Jan 2017 00:00:00 +0000

Why?

We hope that once launched, the ready4 MVP systems model will be transparent, reusable and updatable model that can demonstrate its usefulness for addressing some important topics in youth mental health. However, there will inevitably be a much greater number of topics for which that the MVP model lacks the scope to adequately address. The two main scope limitations of the MVP model are expected to be omissions and level of abstraction. Some relevant system features will be ommitted from representation by the MVP model - for example our pipeline of platforms modules does not currently include any planned modules for modelling the operations of digital mental health services or schools. System features that are represented in the MVP model may only have one level of abstraction, which may be either too simple or too complex to be appropriately applied to some modelling goals.

What?

We plan to progressively extend the scope of ready4 and the range of decision topics to which it can validly be applied. The most important initial types of help we need to achieve this goal are funding, projects and advice.

How?

The two main strategies for extending ready4 are to translate existing models and develop new models. The process for developing new models is outlined elsewhere as the steps required to undertake a modelling project.

Translating existing models involves the following steps:

Identify existing computational model(s) of relevant youth mental health systems to be redeveloped using the ready4 framework. Processes for identifying models could include:

A modelling team reviewing some of the models that they have previously implemented using other software; and/or
A systematic search of published literature and/or model repositories.

(Optional - only if a single project plans to redevelop multiple models) Develop a data extraction tool into which data on relevant model features will be collated and categorised.
Extract data on relevant model features. In the (highly likely) event that the reporting and documentation of the model being redeveloped lacks important details:

Contact the original model authors for assistance; and/or
Seek relevant advice to help determine plausible / appropriate values for missing data.

Author module libraries for representing the included model(s).
Author labelled open access datasets of model input data (which can be set to private for now).
Author analysis and reporting programs designed to replicate the original modelling study / studies.
Compare results from original and replication analyses. Ascertain the most plausible explanations for any divergence between results. Where this explanation relates to an error or limitation in the new ready4 modules or analysis programs that have been authored, fix these issues.
Complete documentation of model libraries, datasets and analyses.
(If not already done) Publish / link to datasets on the ready4 Dataverse and share releases of libraries and programs in the ready4 Zenodo community.

When?

As our current focus in on developing the MVP model, we are not yet actively pursuing this priority. That will change if we are successful in securing more support from funders. In the mean time, if you are a researcher and/or modeller who is interested in leading a project that can help extend ready4, you can contact our project lead for guidance and/or to discuss the potential for collaborations.

Docs: Priority 6: Integrate ready4 with other open source tools

Thu, 05 Jan 2017 00:00:00 +0000

Why?

Currently all ready4 software is developed using the R language. Although R is powerful, popular and flexible, there are limitations to relying on this toolkit alone. For some tasks, tools written in other languages provide superior performance. Requiring coders to have knowledge of R erects barriers to participation that thus the rate and quality of ready4’s development.

What?

We aim to support and integrate the development and use of tools to implement and extend the ready4 model in multiple languages, with an initial focus on python. The most important initial types of help we need with achieving this goal are advice, funding and code contributions.

How?

This is a longer term program of activity that has yet to be planned. We expect the first step in this process will be convening an advisory group of interested stakeholders to help us identify appropriate actions.

When

We have no active plans to progress this during our current 2023-2025 activity cycle. However, we are open to providing whatever support and guidance we can to researchers and organisations who are interested in leading a project of this nature.