Notice2022-20703
Challenge Competition: Announcement of AHRQ Challenge on Integrating Healthcare System Data With Systematic Review Findings
Primary source
Metadata and text below are from the Federal Register, a public-domain U.S. government work. Always verify the official published version before relying on it for any legal matter.
Published
September 26, 2022
Issuing agencies
Health and Human Services DepartmentAgency for Healthcare Research and Quality
Abstract
The Agency for Healthcare Research and Quality (AHRQ) is announcing a challenge competition to explore the resources and infrastructure needed to integrate real-world data from healthcare systems into systematic review findings. This healthcare systems data can augment study findings synthesized in systematic reviews in a number of ways, including by filling evidence gaps identified in the systematic review to strengthen the available evidence, and by examining the applicability of systematic review findings to real-world populations, including population subgroups not examined in published studies. Ultimately, this work will help AHRQ understand if and how sources of data and information outside of traditional systematic reviews, particularly from healthcare systems themselves, could be used alongside systematic reviews to improve healthcare decision making, healthcare delivery and potentially patient outcomes. This challenge competition will start on (September 26, 2022) and will be completed in two phases, with cash prizes awarded at the end of Phase 2 to all of those proceeding to Phase 2 and to the winners of Phase 2. The winner and runner-up from Phase 2 will be posted on the AHRQ website.
Full Text
<html>
<head>
<title>Federal Register, Volume 87 Issue 185 (Monday, September 26, 2022)</title>
</head>
<body><pre>
[Federal Register Volume 87, Number 185 (Monday, September 26, 2022)]
[Notices]
[Pages 58350-58353]
From the Federal Register Online via the Government Publishing Office [<a href="http://www.gpo.gov">www.gpo.gov</a>]
[FR Doc No: 2022-20703]
=======================================================================
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Agency for Healthcare Research and Quality
Challenge Competition: Announcement of AHRQ Challenge on
Integrating Healthcare System Data With Systematic Review Findings
AGENCY: Agency for Healthcare Research and Quality (AHRQ), HHS.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: The Agency for Healthcare Research and Quality (AHRQ) is
announcing a challenge competition to explore the resources and
infrastructure needed to integrate real-world data from healthcare
systems into systematic review findings. This healthcare systems data
can augment study findings synthesized in systematic reviews in a
number of ways, including by filling evidence gaps identified in the
systematic review to strengthen the available evidence, and by
examining the applicability of systematic review findings to real-world
populations, including population subgroups not examined in published
studies. Ultimately, this work will help AHRQ understand if and how
sources of data and information outside of traditional systematic
reviews, particularly from healthcare systems themselves, could be used
alongside systematic reviews to improve healthcare decision making,
healthcare delivery and potentially patient outcomes. This challenge
competition will start on (September 26, 2022) and will be completed in
two phases, with cash prizes awarded at the end of Phase 2 to all of
those proceeding to Phase 2 and to the winners of Phase 2. The winner
and runner-up from Phase 2 will be posted on the AHRQ website.
DATES: Phase 1 Submission Deadline on January 9, 2023 and Phase 2
Submission Deadline on July 10, 2023.
ADDRESSES: Submit your responses electronically via: <a href="https://www.challenge.gov/?challenge=ahrq-challenge-on-integrating-healthcare-system-data-with-systematic-review-findings">https://www.challenge.gov/?challenge=ahrq-challenge-on-integrating-healthcare-system-data-with-systematic-review-findings</a>.
FOR FURTHER INFORMATION CONTACT: Suchitra Iyer, Director, Technology
Assessment Program; Email: <a href="/cdn-cgi/l/email-protection#9edfd6cccfddf6fff2f2fbf0f9fbeddefff6ecefb0f6f6edb0f9f1e8"><span class="__cf_email__" data-cfemail="46070e1417052e272a2a232821233506272e3437682e2e3568212930">[email protected]</span></a>, Telephone: 301-
427-1550.
SUPPLEMENTARY INFORMATION:
Problem Statement
The Agency for Healthcare Research and Quality (AHRQ), U.S.
Department of Health and Human Services (HHS), is announcing a
challenge competition to explore the feasibility, resources and
infrastructure needed to integrate real world data from healthcare
systems into systematic review findings. The statutory authority for
this challenge competition is Section 105 of the America COMPETES
Reauthorization Act of 2010.
AHRQ's Evidence-based Practice Center (EPC) Program produces
systematic reviews which synthesize information from the peer reviewed
literature and provide the state of the science on available healthcare
technologies (such as pharmaceuticals and medical devices) as well as
healthcare delivery strategies. The process of creating these reviews
is stakeholder driven, methodologically rigorous, and transparent.
Reviews are used to inform healthcare decisions, including
recommendations in clinical practice guidelines as well as national
coverage determinations by Medicare. AHRQ also supports healthcare
systems in their efforts to improve the quality of care and optimize
patient outcomes; systematic reviews are scoped to address issues of
priority to healthcare systems. Yet, due to limitations in the
literature base, EPC systematic reviews may be inconclusive or only
represent a narrow patient population, making it difficult to
generalize or implement the findings within heterogeneous healthcare
systems.
Systematic reviews may also lack contextual details that can inform
successful implementation. Improving healthcare delivery (and thus
patient outcomes) often entails addressing issues beyond the benefits
or harms of an intervention, traditionally the objective of a
systematic review. Traditional reviews may not explain gaps in uptake
or use of a clinical service and questions about how best to implement
a given clinical service (e.g., details around implementation of a
service or intervention).
A recent EPC Program methods report (<a href="https://effectivehealthcare.ahrq.gov/products/unpublished-health-data/methods-report">https://effectivehealthcare.ahrq.gov/products/unpublished-health-data/methods-report</a>) articulates specific scenarios with examples of where
healthcare system data may most effectively complement systematic
reviews (i.e., to improve the strength, applicability, and
implementation of evidence). In one example, investigators at the Mayo
Clinic found that published evidence on outcomes following total
pancreatectomy was sparse, so they supplemented a meta-analysis of
published studies with their own unpublished healthcare system data,
which more than doubled the sample size and improved the strength of
evidence available (<a href="https://pubmed.ncbi.nlm.nih.gov/20681992/">https://pubmed.ncbi.nlm.nih.gov/20681992/</a>).
In another instance, secondary analyses of Veterans Affairs (VA)
data (<a href="https://pubmed.ncbi.nlm.nih.gov/35810550/">https://pubmed.ncbi.nlm.nih.gov/35810550/</a>) confirmed the
applicability to the VA of findings from a published systematic review
(<a href="https://www.hsrd.research.va.gov/publications/esp/robot-gen-surg.cfm">https://www.hsrd.research.va.gov/publications/esp/robot-gen-surg.cfm</a>).
The recent EPC methods report also outlines important limitations
and considerations when using unpublished healthcare system data
alongside systematic reviews, such as relevant limitations in data
quality. However, the report did not address the necessary resources,
skills, partnerships, and processes required to utilize healthcare
system data alongside systematic reviews to strengthen the
actionability of systematic reviews.
This Challenge therefore invites applicants to conduct an analyses
of healthcare system data to supplement an existing AHRQ EPC Program
systematic review. This will help AHRQ understand if and how sources of
data
[[Page 58351]]
and information outside of traditional systematic reviews, particularly
from healthcare systems themselves, could be used alongside systematic
reviews to improve decision making, healthcare delivery, and
potentially patient outcomes.
Challenge Goal
The AHRQ EPC Program is interested in learning how analysis of
real-world data collected by healthcare systems can be used in
conjunction with findings from an AHRQ systematic review to inform
healthcare decision-making in the context of a specific local setting.
The goal of this challenge is to explore and determine the
feasibility, resources, and infrastructure needed to incorporate
unpublished healthcare system data into systematic review findings.
Ideally, these data will enable the healthcare system to make decisions
about which practices to incorporate locally and how to overcome
barriers to implement the evidence to improve clinical practice,
healthcare system operations and, potentially, health outcomes. The
winner and runner-up from Phase 2 will be posted on the AHRQ website.
Challenge Design
<bullet> All evidence reports (systematic reviews, rapid reviews,
and technical briefs) published on the AHRQ website since January 2018
(<a href="https://effectivehealthcare.ahrq.gov/about">https://effectivehealthcare.ahrq.gov/about</a>) may be considered for this
Challenge.
<bullet> Healthcare systems and other provider groups interested in
implementing evidence into practice at their sites may apply.
<bullet> An organization may choose to address no more than 2
systematic reviews, submitting a separate proposal for each review.
<bullet> Teams should have expertise in the clinical topic,
evidence-based practice, data analysis, and quality improvement.
This Challenge consists of two phases:
Phase 1: Proposal
Elicit written proposals on a topic for which an AHRQ evidence
report (systematic review, rapid review or technical brief) has been
published since January 2018 (<a href="https://effectivehealthcare.ahrq.gov/about">https://effectivehealthcare.ahrq.gov/about</a>) and that is relevant to a dilemma in the applicant's healthcare
system. Each proposal is to be written in the form of a narrative that:
1. Explicitly states the rationale (i.e., addressing evidence gaps
to strengthen available evidence, examining applicability of findings
to real-world patients) to complement findings from an EPC report with
analysis of health system data, including a discussion of possible
limitations of the analysis.
2. Develops and describes an analytic plan for use of healthcare
system data [EHR data from an individual healthcare system or networks
of healthcare systems (for example, PCORnet, Epic's Cosmos, etc.)].
3. Provides an approach for decision-making based on the results of
the data analysis and the evidence report, and an evaluation of the
decision-making process and results.
4. Describes potential challenges, barriers, and strategies to
successfully complete the analysis.
5. Lists team members, their role, area of expertise and hours on
project.
A total of 5 proposals will be selected as winners for Phase 1.
Phase 2: Analyses
Healthcare systems selected as winners in Phase 1 will be invited
to provide a written narrative that:
1. Includes a complete analysis of internal real-world data and
appraisal of the analysis for risk of bias using a formal tool such as
the ROBINS I (<a href="https://www.bmj.com/content/355/bmj.i4919">https://www.bmj.com/content/355/bmj.i4919</a>).
2. Specifies how the findings from unpublished data support,
refute, and/or otherwise complement findings from the published
evidence examined in the systematic review. If the unpublished data
conflicts with the AHRQ review's conclusions, discuss possible reasons
for the discrepancy [e.g., challenges with internal validity of
healthcare system data analysis related to study design and methods
used, or challenges with external validity with respect to population
sub-groups (gender, race/ethnicity, multimorbidity) examined in the
healthcare system data].
3. Describes how the unpublished data informed decision-making
(e.g., adapt, adopt, abandon, prioritize).
4. Reports on solutions to any barriers encountered to using
healthcare system data alongside a published evidence review, including
barriers with access to healthcare system data, interoperability of
data sources, and data analysis.
5. Briefly describes potential approaches to implement or
deimplement the evidence, including use of clinical advisories,
clinical pathways, clinical decision support, or any other method. The
plan should describe anticipated risks and barriers and strategies for
successful implementation. Decisions made against uptake should be
justified.
Timeline and Prize Amounts
AHRQ is hosting this challenge as a two-phase competition. All
costs associated with developing and submitting proposals as well as
conducting the analysis of real-world data will be the responsibility
of the Challenge participant. Cash prizes will be awarded only after
the projects are evaluated and determined acceptable at the end of
Phase 2.
Timeline
<bullet> September 26, 2022--Challenge launch.
<bullet> January 9, 2023--Submissions for Phase 1 (written
proposals) are due. AHRQ will complete the review of the proposals
within 6-8 weeks of closing the announcement.
<bullet> March 10, 2023--AHRQ will announce the Phase 1 winners.
Phase 2 of the Challenge will commence once the Phase 1 winners are
announced and notified by March 10, 2023. The AHRQ team will schedule a
live, virtual technical assistance webinar with all winners of Phase 1
to discuss scope of content, accessibility/compliance with Section 508,
and address questions that the winners may have.
<bullet> March 10, 2023--Phase 2 participants will have at least
120 calendar days from notification to conduct and submit their
analyses as described in their proposal(s). The deadline to submit the
analysis is July 10, 2023.
<bullet> Fall 2023--The final winners of Phase 2 of the competition
will be announced in October 2023.
Prize Amounts
The top five entries in phase one will be invited to participate in
Phase 2. Upon completion of Phase 2, each of the top five entries will
each receive cash awards of $50,000. Additionally, the first-place
winner from Phase 2 will be awarded an additional $150,000 and the
runner-up will be awarded an additional $100,000. The winner and
runner-up from Phase 2 will be posted on the AHRQ website.
How To Enter the Challenge
Participants can register by visiting the <a href="http://Challenge.gov">Challenge.gov</a> website
(<a href="https://www.challenge.gov">https://www.challenge.gov</a>). Participants should carefully review
challenge information and submission requirements on the website,
including the intellectual property rules and assessment criteria.
Participants are encouraged to follow the Challenge on <a href="http://Challenge.gov">Challenge.gov</a> to
obtain any updates and reminders of upcoming deadlines.
Submission Requirements
Phase 1
The submitted proposals must be written in US English and submitted
[[Page 58352]]
using <a href="http://Challenge.gov">Challenge.gov</a> no later than January 9, 2023. Applicants or
applicant organizations shall submit no more than 2 proposals, and no
proposal shall describe more than one topic. Each proposal will be no
more than 2,000 words, double spaced, 11-point type size, with 1-inch
margins. Include in proposals plans for meeting Section 508
accessibility standards.
Phase 2
Analyses shall be a journal article-style written narrative in U.S.
English in no more than 4,000 words and submitted using the
<a href="http://Challenge.gov">Challenge.gov</a> website no later than July 10, 2023.
Review Process
All submissions will be reviewed by at least two subject matter
experts from within or outside the federal government who will score
the proposals based on the assessment criteria and provide a brief
comment about the submission. The scores/comments on Phase 1 and Phase
2 submissions will be compiled, and a ranked summary provided to AHRQ
staff. AHRQ will select winners based on quantitative and qualitative
assessments.
Evaluation Criteria for Selecting Winning Applications
Assessment Criteria for Phase 1
1. Approach (40 points)
Does the proposal sufficiently describe:
a. the context and rationale for why the EPC report has been
chosen, and how the analyses of unpublished data will complement the
findings from the systematic review?
b. the data source and the analytic approach? Does the described
analytic approach provide the right balance of feasibility, rigor, and
innovation for the project?
2. Impact (40 points)
Does the proposal clearly and concisely describe:
a. potential routes for uptake of evidence within the healthcare
system?
b. method to measure whether the uptake will have an impact on
healthcare delivery, quality of care or patient outcomes?
c. anticipated risks, barriers, and strategies to successfully
complete the project?
3. Team (20 points)
Does the team have the right combination of expertise to support
the proposed technical approach?
Compliance (pass/fail)--Does the Phase 1 proposal adequately
address required accessibility standards (Section 508)?
Assessment Criteria for Phase 2
Participants in Phase 2 may be disqualified if their submitted
analyses deviate from their Phase 1winning proposals.
1. Analysis (40 points)
a. Are the description and discussion of the findings from the
analysis comprehensive?
b. To what degree is the alignment/nonalignment between healthcare
system data, the AHRQ systematic review, and the healthcare system's
decision-making explained?
2. Description of the Process (40 points)
a. Are the risks, barriers, challenges encountered, solutions, and
required infrastructure well described?
3. Future Plans (20 points)
a. Is the preliminary plan for implementation and evaluation
appropriate and feasible?
Compliance (pass/fail)--Does the Phase 1 proposal adequately
address required accessibility standards (Section 508)?
Eligibility Rules for Participating in the Challenge
To be eligible under this Challenge, an individual (whether
participating singly or in a group) or entity:
1. Shall have registered (<a href="http://Challenge.gov">Challenge.gov</a>) to participate in the
Challenge.
2. Shall have complied with the rules set forth in this
announcement for participation in this Challenge.
3. Shall be incorporated and maintain a primary place of business
in the United States (in the case of a private entity), and in the case
of an individual, whether participating singly or in a group, shall be
a citizen or permanent resident of the United States.
4. May not be a Federal entity or Federal employee acting within
the scope of their employment. (All Federal employees should consult
with their agency Ethics Official to determine whether the federal
ethics rules will limit or prohibit the acceptance of a prize).
5. May not be an employee of AHRQ or any other company,
organization, or individual involved with the design, production,
execution, judging, or distribution of the Challenge, or their
immediate family (spouse, parents and step-parents, siblings and step-
siblings, and children and step-children), or household members (people
who share the same residence at least 3 months out of the year).
6. May not use Federal funds from a grant to develop Challenge
applications unless consistent with the purpose of the grant award.
7. May not use Federal funds from a contract to develop Challenge
applications or to fund efforts in support of a Challenge submission.
8. Shall not be deemed ineligible because the individual or entity
used Federal facilities or consulted with Federal employees during a
competition if the facilities and employees are made equitably
available to all individuals and entities participating in the
competition.
9. Shall not be required to purchase liability insurance as a
condition of participation in this competition.
Additional Rules of Participation
By participating in this Challenge, each individual (whether
participating singly or in a group) or entity:
1. Agrees to follow all applicable Federal, State, and local laws,
regulations, and policies.
2. Agrees to comply with all terms and conditions of participation
in this Challenge.
3. Agrees that the submission will not use HHS or AHRQ logos or
official seals, except as authorized by HHS or AHRQ. Notwithstanding
this authorized use of AHRQ/HHS branding, participants will not claim
endorsement by AHRQ/HHS.
4. Understands that all materials submitted to AHRQ as part of a
submission become AHRQ records. Any confidential commercial or
financial information contained in a submission must be clearly
designated as such at the time of submission.
5. Agrees that a winning submission may only be announced by AHRQ
and in a time, place, and manner determined by AHRQ in its sole
discretion. Winners will be permitted to publicize and publish their
winning submissions in accordance with instructions provided by AHRQ.
The winner and runner-up from Phase 2 will be posted on the AHRQ
website.
6. Agrees that the submission must not infringe upon copyright or
any other rights of any third party.
7. Agrees to assume any and all risks and waive claims against the
Federal Government and its related entities, except in the case of
willful misconduct, for any injury, death, damage, or loss of property,
revenue, or profits, whether direct, indirect, or consequential,
arising from participation in this prize contest, whether the injury,
death, damage, or loss arises through negligence or otherwise.
8. Agrees to indemnify the Federal Government against third-party
claims
[[Page 58353]]
for damages arising from or related to Challenge activities.
9. Understands that circulation of findings could be worldwide, and
that the Federal Government will not compensate the participants for
any use; winners shall receive a one-time cash prize as set forth in
this announcement. The winner and runner-up from Phase 2 will be posted
on the AHRQ website.
10. Understands that AHRQ reserves the right to cancel, suspend,
and/or modify this prize contest, or any part of it, for any reason, in
AHRQ's sole discretion. AHRQ also reserves the right not to award any
prizes if no entries are deemed worthy.
11. Understands that AHRQ will not select a winner that is named on
the Excluded Parties List System (EPLS).
Intellectual Property (IP) Rights
1. Each participant retains title and full ownership in and to
their submission. Participants expressly reserve all intellectual
property rights not expressly granted.
2. By participating in the Challenge, each participant (whether
participating singly or in a group) acknowledges that he or she is the
sole author or owner of, or has a right to use, any copyrightable works
that the submission comprises, that the works are wholly original with
the participant (or is an improved version of an existing work that the
participant has sufficient rights to use and improve), and that the
submission does not infringe any copyright or any other rights of any
third party of which participant is aware.
3. In addition, each participant (whether participating singly or
in a group) grants to the U.S. Government a paid-up, nonexclusive,
royalty-free, irrevocable worldwide license in perpetuity, and the
right to reproduce, publish, post, link to, share, display publicly (on
the web or elsewhere) and prepare derivative works, including the right
to authorize others to do so on behalf of the U.S. Government.
4. Each participant must clearly delineate any intellectual
property and/or confidential commercial information contained in a
submission that the participant wishes to protect as proprietary data,
in accordance with Additional Rules of Participation No. 4.
5. If the submission includes any third-party works (such as third-
party content or open-source code), the participant must be able to
provide, upon request, documentation of all appropriate licenses and
releases for use of such third-party works. If the participant cannot
provide documentation of all required licenses and releases, AHRQ
reserves the right, in its sole discretion, to disqualify the
submission.
Dated: September 20, 2022.
Marquita Cullom,
Associate Director.
[FR Doc. 2022-20703 Filed 9-23-22; 8:45 am]
BILLING CODE 4160-90-P
</pre><script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script></body>
</html>Indexed from Federal Register on September 26, 2022.
This is legal information, not legal advice. Laws vary by jurisdiction and change frequently. Always verify current law with official sources and consult a licensed attorney in your jurisdiction for advice on your specific situation.