Skip to main contentSkip to footer content

Tracking and Evaluation Core

Brian Melancon

Brian Melancon
Core Director
Pennington Biomedical Research Center
(225) 763-2980

Contact this core
Email: tracking@lacats.org

 

About This Core

The Tracking and Evaluation (T&E) Core is charged with developing and implementing strategies, processes and activities to collect useful data and provide meaningful evaluations at both the Core-level and overall Center-level. The core is lead out of Pennington Biomedical Research Center. 

The goal (specific aim) of this core is to strengthen and oversee the execution of an internal tracking and evaluation plan that ensures ongoing monitoring, informative interim assessments, flexibility to pivot priorities, and an annual evaluation at both the Core-level and Center-wide level for new and continuing initiatives.

Measuring & Evaluating Progress 

Evaluations will follow the logic model, supplemented by the overarching progress indicators. Formative assessments, conducted quarterly, will focus on review of Core milestones and other process and outputs evaluation data to determine if the Core is on course or is facing an unexpected challenge (Figure 2). The T&E team meets with the PI and Administrative Core weekly to discuss recent progress and known challenges. This feedback loop allows the PI to take quick next steps as needed. T&E also provides a quarterly report to the PI and the PI, in turn, shares the report as part of the PI meetings with the IAC. Data sources include SPARC, REDCap, PubMed, NIH Reporter, Core reports (Cores submit reports each quarter to reflect annual progress to-date), informal one-on-one and Operations Committee discussions. The annual Core-level summative evaluation is a process that includes the summation of quarterly assessments, applicable plans for correction or changes, Core plans for the upcoming year; an independent evaluation by the External Advisory Committee (EAC); and the PI evaluation of Core leader performance.

The Center’s effectiveness is built by the success of each Core to provide products or services that support the Center’s overarching specific aims, and that are valuable to our institutions, investigators, and the population we serve. We will follow the logic model supplemented by the overarching progress indicators, to track/evaluate center-wide progress toward achieving the overarching Aims (Table 2) and Five-Year Milestones.

Formative assessments for the Center are based on OC and Steering Committee discussions, informal phone calls and meetings with the PI, Co-PDs, Core leaders and personnel, and data collected via SPARC, REDCap, PubMed, NIH RePORTER, user feedback as well as the Core level assessments. The T&E team meets with the PI weekly  to facilitate a quick feedback loop and identify immediate next steps.

Annual summative evaluation includes the independent EAC evaluation. The T&E team will follow the logic model supplemented by the overarching progress indicators. The focus will be on assessing: a) organizational strengths and progress toward achieving the LA CaTS four overarching specific aims and five-year milestones; b) use, quality and costs of programs, resources, and services; budget management, resource allocation, program development or improvement, challenges; c) growth in CT research focused on underserved, vulnerable populations and addressing health disparities; and, d) identifying the results of LA CaTS Center work with the IDeA-CTRs, CTSAs and others to move national goals forward. Emphasis will be placed on assessing new practices or methods initiated by the LA CaTS Center for delivering programs, resources, and services.

The LA CaTS Center metrics and progress indicators are based on our External Advisory Committee recommendations, the direction of our funder the NIH-NIGMS and the needs of the state as identified by the LA CaTS Center to address health disparities and improve health outcomes. The progress objectives are summarized in the below graphic:

Indicators

Measure #/type of scientific product and other units of related information required
by NIH-NIGMS.

Scientific products include manuscripts, publications, presentations, posters, external grant submissions and awards, intellectual property activity and completed scientific advance forms submitted annually to NIH-NIGMS.

Measures:

  1. #/type of investigator and study characteristics reported in SPARC (e.g., name, degree, volunteered demographics; study title, disease state(s) or impact area focus; T0-T4 classification);
  2. CTU activity - #/type/title/name of protocols supported in the CTUs each quarter; #/type, activity of study participants in CTUs each quarter;
  3. #users/type/results of internal competitive funding/training opportunities;
  4. #/type of training and/or professional development opportunities that are open to all member institutions and other stakeholders;
  5. CE Core infrastructure support for research, training, collaborating, and communicating;
  6. #/type demographics related to diversity member institution faculty, students, personnel using LA CATS resources.

Measures:

  1. Collaboration: title/type, names of collaborators, reported results (collaboration types listed below);
  2. #/type Investigators use of Core(s) and Core(s) utilized;
  3. CE Core support for virtual collaborative, networking, communication, and education opportunities. Collaboration types include a) multi-disciplinary, multi-institution, mentor-mentee teams, others; b) Core collaborations with other Cores, and/or with IDeA-CTRs, CTSAs, COBRE, INBRE, FQHCs, rural clinic providers, and others; c) Cross-member institution collaborations (e.g., resource coordination).

Measures:

  1. Regulatory processes will include: a) Internal research/project funding programs: Time (days) between (i) date of notice of award sent to an investigator/scholar and the date of the final applicable regulatory process approval; (ii) date of final regulatory approval to the study start date approved by LA CaTS. b) NIH Facilitated Review models: (i) #/type/name of institution and role; (ii) time from submission to approval if available (e.g., lead or relying and based on information received from the national coordinator).
  2. Internal competitive funding/training applications review process for each round/cohort: time (days) between a) deadline date for the round of applications to be submitted and start date for first application to be sent for review; b) start date for the first application review to the end of the last application review; c) date of the end of review process to award announcement.
  3. No Cost Extensions: a) #/type of No Cost Extension Requests; (b) #/type NCE requests approved; (c) #/type of projects that do not submit a final report by the date approved in the NCE; (d) productivity as a result of the NCE.

Measures:

  1. Type/results of member institution initiatives that coordinate with this LA CaTS Center focus, which includes the issue of disparity in access to health care facilities or providers;
  2. #/type/results of research studies with this focus as self-reported by investigators in SPARC;
  3. # of publication titles or other scientific product titles indicating this focus;
  4. #/type/results of Core-to-Core collaborations or individual Core initiatives with this focus;
  5. Type of health and societal benefits reported as a result.

Measures:

  1. Community and public health benefits;
  2. Clinical and medical benefits;
  3. Policy and legislature benefits;
  4. Economic Benefits

Measures:

  1. Type/results of technical/logistical institutional support, e.g., speaker support for
    webinars/seminars; space for seminars/meetings; service on applicable committees;
  2. #/type/result of crossinstitutional collaborations focused on community engagement and/or health literacy initiatives; and
  3. #/type/result of LA CaTS member institutions providing internal institutional communication support.

SPARC is our web-based central portal for requesting services & resources across the LA CaTS Center that was developed by the Medical University of South Carolina CTSA. Valuable information is tracked from our investigators using SPARC. The information gathered is used for NIH reporting, advisory committees, and overall improvements in efficiencies and reach of our investigators we serve. SPARC is a cross-core collaborative effort between the T&E and Cyberinfrastructure Enhancement  Cores. See SPARC here.

This guide combines NIH’s RPPR and NIGMS’s updated reporting guidelines for CTRs.

NIH and NIGMS Reporting GuidelinesThis document was update in July 2024.