24-106 Evaluation Researcher

Colorado State University, United States

Updated: 4 days ago
Location: Boulder, COLORADO
Deadline: 29 Apr 2024

Posting Details
Posting Detail Information


Working Title 24-106 Evaluation Researcher
Position Location Boulder, CO
Work Location Position qualifies for hybrid/in-office work
Research Professional Position Yes
Posting Number 202400558AP
Position Type Admin Professional/ Research Professional
Classification Title Research Associate III
Number of Vacancies 1
Work Hours/Week 40
Proposed Annual Salary Range $110,000 - $115,000 Salary is commensurate with experience and qualifications.
Employee Benefits
Colorado State University is not just a workplace; it’s a thriving community that’s transforming lives and improving the human condition through world-class teaching, research, and service. With a robust benefits package, collaborative atmosphere, commitment to diversity, equity and inclusion , and focus on work-life balance, CSU is where you can thrive, grow, and make a lasting impact.
  • Review our detailed benefits information here.
  • Explore the additional perks of working at CSU here .
  • For the total value of CSU benefits in addition to wages, use our compensation calculator .
  • Lastly, click here for more information about why Fort Collins is consistently ranked in the top cities to live in !
Desired Start Date 06/03/2024
Position End Date (if temporary)
To ensure full consideration, applications must be received by 11:59pm (MT) on 04/29/2024
Description of Work Unit
The Cooperative Institute for Research in the Atmosphere (CIRA) at Colorado State University seeks to fill a social science research and evaluation position with the National Oceanic and Atmospheric Administration (NOAA) located at the David Skaggs Research Center (DSRC), Global Systems Laboratory (GSL) in Boulder, CO.
The position will be for a social science researcher with a background in qualitative and quantitative social science research and evaluation methods to serve as the Evaluation Researcher for the Fire Weather Testbed (FWT) within GSL. The Evaluation Researcher will design and implement experiments and evaluations in the FWT of new and emerging fire weather products, information, services, technologies, and communication techniques. This will include evaluating how, if at all, weather forecasters use new and existing fire weather and smoke products, how these products support current and future decision-making paradigms, and how these new products and services support risk communication to end-users.
This position sits within the Weather Informatics and Decision Support (WIDS) Division as part of the interdisciplinary FWT team. The incumbent’s office will be in Boulder, CO at a Federal building and requires the ability to pass a National Agency Check with Inquiries (NACI, Tier 1 federal background check). An onsite presence at the GSL in Boulder, CO, with an option for part-time telework, is required for this position.
The Cooperative Institute for Research in the Atmosphere (CIRA) at Colorado State University (CSU) is a multi-million dollar research organization located on CSU’s Foothills Campus in Fort Collins, Colorado. CIRA is a cooperative institute that is also a research department within CSU’s College of Engineering, in partnership with the Department of Atmospheric Science. Its vision is to conduct interdisciplinary research in the atmospheric sciences by entraining skills beyond the meteorological disciplines, exploiting advances in engineering and computer science, facilitating transitional activity between pure and applied research, leveraging both national and international resources and partnerships, and assisting NOAA, Colorado State University, the State of Colorado, and the Nation through the application of our research to areas of societal benefit.
NOAA’s Global Systems Laboratory (GSL) is a federal science and research laboratory under NOAA’s Office of Oceanic and Atmospheric Research. GSL provides the National Weather Service (NWS) and the nation with environmental observing, prediction, computer, visualization, and information systems. These systems deliver data, forecasts, and predictions of weather, including severe weather events, within the next few minutes to weeks away. GSL is a leader in the applied research, directed development, and technology transfer of environmental data, models, products, and services that enhance environmental understanding with the outcome of supporting commerce, protecting life and property, and promoting a scientifically literate public.
The FWT is a new NOAA testbed dedicated to serving the wildfire communities through pursuing improvements to fire weather forecasting and decision support services. These improvements will be achieved by connecting stakeholders and their information needs to the scientific research and development communities. The FWT is a physical and virtual environment designed to bring the wildfire community together, with an end goal of a safer society and healthier environment achieved through scientific and technological advances. Numerous experiments will be conducted in the FWT and will include a broad spectrum of science fields including atmospheric, geophysical, hydrologic, and environmental sciences as well as social and behavioral sciences. The Evaluation Researcher will be expected to design and implement a wide variety of evaluations from surveys, focus groups, and virtual evaluations to week-long in-person evaluations and field experiments.
Position Summary
CIRA is seeking a social science researcher with a background in qualitative and quantitative social science research and evaluation methods. The successful candidate will design and implement evaluations in the FWT from a human-centric and usability perspective. This will include evaluating how weather forecasters (NWS Forecasters, Incident Meteorologists and other forecasters) use new and/or existing information, tools, and products, and how that information is communicated to and used by (but not limited to) emergency managers, fire managers and practitioners, incident commanders, government safety officials, and other users.
The Evaluation Researcher is expected to design, implement, and report on up to a dozen or more wide-ranging evaluations in the FWT each year. The Evaluation Researcher will work with Principal Investigators, product developers, and the FWT team to identify questions pertaining to usability and user-needs to ensure that evaluations are designed to effectively answer those questions. The researcher will also collaborate closely with the User Needs Assessment Team of the FWT and GSL’s Social and Behavioral Sciences (SBS) Branch about ongoing fire weather-specific social science research.
The Evaluation Researcher will provide subject-matter expertise in the areas of mixed-method, qualitative and quantitative social science. This includes, but is not limited to, evaluation research, workflow and/or task load analysis, and/or decision analysis. The Evaluation Researcher will be responsible for working closely with the other members of the WIDS SBS Branch, as well as meteorologists and developers. Evaluation topics will be driven by our key partners through the FWT user-needs assessment, ranging from process evaluations of new fire weather applications, product development and refinement, and usability assessments of various fire weather products and services.
The Evaluation Researcher will be expected to lead the development of evaluation reports following each evaluation, as well as scientific publications and presentation of findings at scientific meetings and conferences. This position works very closely with the Federal FWT Manager/Lead Scientist and the SBS Branch Chief, and formally reports to the CIRA Social and Behavioral Scientist. This position will be assigned a classification of Research Associate III or IV based on education and experience as listed in the required qualifications.
Required Job Qualifications
In your cover letter, please specifically address EACH requirement as relates to your experience. A cover letter that fails to address the required qualifications for this position may not be considered further after review by the search committee.
RA III Requirements
  • M.S. in a relevant field* described below plus
    • Two (2) years of related professional experience.


OR—
  • Ph.D. in a relevant field* described below.

* Formal education required in a relevant social or behavioral science or interdisciplinary field such as, but not limited to: evaluation, risk communication, decision science, psychology, sociology, or related field.
RA IV Requirements
  • M.S. in a relevant field* described below plus
    • Five (5) years relevant work experience.

OR–
  • Ph.D. in a relevant field* described below plus
    • Two (2) years relevant work experience.

*Formal education required in a relevant social or behavioral science or interdisciplinary field such as, but not limited to: evaluation, risk communication, decision science, psychology, sociology, or related field.
Required qualifications for both classifications
  • Ability to pass a National Agency Check with Inquiries (NACI, Tier 1 federal background check), due to the position being located inside a Federal facility;
  • The successful candidate must be legally authorized to work in the United States by the start date. CIRA will not sponsor a visa for this position now nor at any point in the future.
  • Well-developed knowledge of and experience employing qualitative and/or quantitative social science and evaluation research methods, including evaluation design, evaluation implementation, collection and monitoring of data, analysis of data, and evaluation reporting. This includes (but is not limited to): at least two (2) years experience conducting qualitative and/or quantitative human subjects social science and evaluation data collection and analysis, including but not limited to, focus groups, workshops, and other methods for gathering participant feedback.
  • Leadership and/or project management experience with research and evaluation projects.
Preferred Job Qualifications
In your cover letter, please specifically address the applicable preferred qualifications for this position. A cover letter that fails to address the preferred qualifications for this position may not be considered further after review by the search committee. 
  • Familiarity with Institutional Review Board training and regulations (including CITI certification/training).
  • General understanding of hazardous weather phenomena.
  • General familiarity with the types of end-users of fire weather information (e.g., Firefighters, Incident Management Teams, Emergency Managers, ect.)
  • Experience building and fostering relationships with end-users.
  • Demonstrated ability to work independently and as part of an interdisciplinary research team.
  • Interest in the interactions between weather information and weather-related decision making in the context of public safety and wildfires.
  • Effective written and oral communication skills, both within field of expertise and across disciplinary boundaries.
Diversity Statement
Reflecting departmental and institutional values, candidates are expected to have the ability to advance the Department’s commitment to diversity and inclusion.

Essential Duties


Job Duty Category Preparation and Design of Evaluations
Duty/Responsibility
  • Develop measurable evaluation questions specific to each evaluation in collaboration with product developers and the FWT team. Design evaluations of new fire weather-related tools, technologies, products, services and end-user risk communication from the forecaster to the end-user.
  • Conduct relevant literature reviews to ensure evaluation designs, tools and approaches are aligned with best practice in the field.
  • Lead and collaborate on developing qualitative (e.g., interview protocols) and quantitative (e.g., surveys) data collection instruments (and associated data management and archival plans) to evaluate how the real-time use of new weather-related forecast technologies, information, products and services are integrated by operational weather forecasters who provide fire weather forecasts and decision support services.
  • Lead and collaborate on developing human-centric research approaches for assessing how new fire weather products, information, and services are communicated to different stakeholder groups (e.g., emergency managers, fire managers, incident commanders, government safety officials) and how that information helps them perform their job duties (e.g., observing emergency managers’ evacuation decisions).
Percentage Of Time 55
Job Duty Category Conducting and Analyzing Evaluations
Duty/Responsibility
  • Co-lead, or assist in the leading of FWT evaluations using qualitative and quantitative evaluation techniques as referenced above (virtual and/or in-person), analyzing collected data, synthesizing findings, codebook development, and communicating findings (to include scientific publications) to diverse audiences (e.g., Lab Leadership, OAR Leadership, scientific presentations, written reports, etc.).
  • Conduct human-computer interaction usability evaluations of users interacting with products, services, systems, or other information and collect behavioral data using techniques and methods common to academic and industry, including but not limited to: one-on-one and focus group interviews, experimental survey design, cognitive workload analysis, eye tracking, mental modeling, and cognitive task analysis.
  • Collaborate with the SBS Branch and FWT User Needs Team to identify themes and areas for research exploration as a result of evaluation efforts, as well as areas for improvement in the development of FWT experiments and evaluations.
  • Work alongside SBS researchers, FWT User Needs Team, and developers to co-interpret and share evaluation results, implications, and next steps for additional research and future FWT activities.
Percentage Of Time 20
Job Duty Category Documentation and Reporting
Duty/Responsibility
  • Document findings and recommendations from each evaluation and co-write evaluation reports with the FWT Team.
  • Present results and findings to internal audiences (e.g., coworkers, Leadership, etc.) and/or external audiences at workshops and professional meetings and conferences (e.g., the American Meteorological Society, American Evaluation Association, and local, regional, or national fire weather-related conferences).
Percentage Of Time 25

Application Details


Special Instructions to Applicants
Applications must be submitted via online portal. We will not accept materials sent via email or other mode. NOTE: In your cover letter, please specifically address the required and preferred qualifications of this position. A cover letter that fails to address the qualifications of this position may not be further considered after review by the search committee. Likewise, an online application with a generic cover letter or missing a cover letter, and/or an application that does not include current/correct reference contact information may not be further considered after review by the search committee. Applications will be accepted until the position is filled; however, to ensure full consideration applications should be submitted by 11:59 PM MT on Monday, April 29, 2024.
CSU is committed to full inclusion of qualified individuals. If you need assistance or accommodations with the search process, please reach out to the listed search contact.
To apply, please upload a cover letter that addresses the required and preferred job qualifications, a resume, and the contact information for three professional references. References will not be contacted without prior notification to candidates.
Conditions of Employment Pre-employment Criminal Background Check (required for new hires), Special Requirements/Other - Ability to pass a National Agency Check with Inquiries (NACI, Tier 1 federal background check), due to the position being located inside a Federal facility; Must be legally authorized to work in the United States by the start date. CIRA will not sponsor
Search Contact [email protected]
EEO Statement
Colorado State University is committed to providing an environment that is free from discrimination and harassment based on race, age, creed, color, religion, national origin or ancestry, sex, gender, disability, veteran status, genetic information, sexual orientation, gender identity/expression, or pregnancy in its employment, programs, services and activities, and admissions, and, in certain circumstances, marriage to a co-worker. The University will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Colorado State University is an equal opportunity and equal access institution and affirmative action employer fully committed to achieving a diverse workforce and complies with all Federal and Colorado State laws, regulations, and executive orders regarding non-discrimination and affirmative action. The Office of Equal Opportunity is located in 101 Student Services.
The Title IX Coordinator is the Director of the Office of Title IX Programs and Gender Equity, 123 Student Services Building, Fort Collins, CO 80523-0160, (970) 491-1715, [email protected] .
The Section 504 and ADA Coordinator is the Director of the Office of Equal Opportunity, 101 Student Services Building, Fort Collins, CO 80523-0160, (970) 491-5836, [email protected] .
The Coordinator for any other forms of misconduct prohibited by the University’s Policy on Discrimination and Harassment is the Vice President for Equity, Equal Opportunity and Title IX, 101 Student Services Building, Fort Collins, Co. 80523-0160, (970) 491-5836, [email protected] .
Any person may report sex discrimination under Title IX to the Office of Civil Rights, Department of Education .
Background Check Policy Statement
Colorado State University strives to provide a safe study, work, and living environment for its faculty, staff, volunteers and students. To support this environment and comply with applicable laws and regulations, CSU conducts background checks for the finalist before a final offer. The type of background check conducted varies by position and can include, but is not limited to, criminal history, sex offender registry, motor vehicle history, financial history, and/or education verification. Background checks will also be conducted when required by law or contract and when, in the discretion of the University, it is reasonable and prudent to do so.

Similar Positions