College Survey Policy

Purpose

Survey research is an ubiquitous tool for collecting vital information from campus constituents. This policy aims to: encourage coordinated efforts and best practices in survey research to produce quality data, create opportunities for cross-collaboration and information-sharing, and promote best practices in information security and privacy.

Scope

This policy applies to survey activities that are of an institutional nature, and not listed below. It applies to all students, faculty, staff, and non-Occidental researchers or agencies seeking to survey the whole, or any part of, the Occidental student, faculty, staff, parent, and/or alumni bodies. For the purposes of this policy, a survey is defined as information (qualitative or quantitative) gathered from human subjects using questionnaires, interviews, polls, evaluations, etc. to make inferences about a target population. An institutional survey is one that seeks to inform priorities, goals, compliance, practices, policies, procedures, operations, or other decision-making processes of the College (see Appendix for reference).

This policy does not apply to the following activities:

(a) faculty research*

(b) faculty-supervised student research being performed as part of an academic course or for academic credit^[1]*                 

(c) teaching evaluation forms

(d) forms used to collect information for administrative purposes (e.g. scheduling)

(e) evaluation of an event by participants

(f) feedback from clients at the point of service

(g) assessments or surveys sent by departments (academic or administrative), program leaders or student group/organization leaders to members within that department, program or group/organization for evaluation or assessment of that department, program, or group/organization (for example, a registered student organization may freely poll its own members or a union may survey its own members).

Policy and Institutional Decision Making

The President of the College, the senior administrative team, and the Office of Institutional Research (IR) are in mutual agreement that survey data can be instrumental in informing decision makers on College policy, practices, procedures or operations. In order for survey data to be considered by decision makers (and the key stakeholders listed above), data provided must be from surveys that adhere to this policy and/or are approved by Institutional Research (where noted below). Data presented from surveys that do not adhere to this policy will be subject to review before determining its significance in the context of institutional decision making. 

Survey projects meant to fulfill requirements from external governing entities, such as federal compliance offices, must also adhere to this policy as a measure of quality assurance.  

Any project, from any community member or group, is encouraged to contact the Office of Institutional Research to determine if the activity is subject to this policy, or would benefit from the policy. All are encouraged to follow these protocols and policies as best practices.

Procedure

Those interested in administering a survey are expected to be able to communicate reasons for needing a survey, what the survey will be used for, what will be collected on the survey, and who it should go to. The following steps outline the procedure for embarking on an institutional survey project. 

Step 1. Planning

Those interested in administering a survey are expected to be able to communicate reasons for needing a survey, what the survey will be used for, what will be collected on the survey, and who it should go to. Familiarize yourself with the proposal form in the planning stage to prepare what you will need. It is encouraged to share this information with others to seek feedback, coordination, collaboration, or endorsement (though this process will formally occur in Step 2). This may include colleagues, supervisors, stakeholders, governing bodies, or leadership and may help inform the research or expedite the proposal review.

The Office of Institutional Research will accept a proposal at any stage of planning process and can assist in development. It is encouraged to communicate with IR early in the planning process, as data or existing surveys may alleviate the need for a survey. Before submitting a proposal, become familiar with these policies and protocols. They outline expectations and will help inform decisions. 

Step 2. Submit Proposal

When ready to initiate a survey project under this policy, submit a survey proposal to the Office of Institutional Research. The following information is requested as part of the proposal (the full proposal form is available here). The proposal does not have to have all information included when submitted, but all information will need to be known before a final review can be completed. Additional materials and information may be requested.

• Description of the project, including content, purpose and intended use of results
• Specific population receiving the survey
• Method of survey distribution (e.g. online, paper, via email, in class, at event, etc.), including (if online) survey software (the College-supported Qualtrics Survey Software is recommended)
• Time frame for administering the survey, including beginning and end dates
• Description of any planned incentive program for respondents
• Current draft of the instrument and all invitation and cover letters
• Description of any personal or identifying information collected, how anonymity or confidentiality will be ensured, and how the data will be stored.

Step 3. Proposal Review

The Office of Institutional Research will review the proposal to ensure best practices in survey design, recruitment strategies, administration methods, human subjects protection, data security; reduce duplications, overlap or over-sampling; and coordinate schedules of data collection with other researchers at Oxy. This step will also allow IR to determine if additional coordination opportunities exist, if the proposed survey can be combined with other planned surveys, or if there are other data available that will allow the survey to be avoided^[2].

The process may include passing the project on for higher level review; communicating revisions, edits, or changes; sharing the proposal with stakeholders for content expertise or collaborative opportunities; asking for clarifying or additional information. This process may repeat several times before the survey is approved. Approval of a survey is based on the following:

• Extent to which other surveys or data sources supply or could supply the required data.
• Extent of the survey burden on target groups within the College, and timing of other survey commitments (timeline alterations may be required to avoid conflicts).
• How the data will be used, the extent to which data will be shared, and the data management plan for securing data and protecting privacy.
• The adherence to the rights of participants and protection of participant privacy and/or confidentiality.
• Usefulness of the data and its relevance to the College’s strategic planning and priorities.
• The design and content of the survey instrument and proposed survey administration methods.

Timeline: Proposals will be reviewed within 3 business days of receipt and IR will communicate next steps. The time it takes for a survey to complete the protocol varies based on several factors, from stage in the planning process, the scope and scale of the survey, the number of reviews, to the amount of revisions (and subsequent views) needed. The sooner the information is complete, the faster the review process can be. For example, Institutional survey projects (see Appendix) should expect to take longer (1-2 months) than procedural surveys (fully reviewed in less than a week). Advocacy surveys should expect the review process to take 1-4 weeks.

Once the Office of Institutional Research (and any other involved parties) have completed the proposal review, the proposal moves to Step 4.

Step 4. Final Approval

If the Office of Institutional Research determines further levels of review are necessary (a typical scenario), IR will identify and seek a final review from key stakeholders at appropriate review levels (see Appendix). This step ensures the feedback loop on coordination, collaboration, and awareness is closed. Individuals requested to give final reviews may request additional revisions, edits, changes, or otherwise. When final reviews are complete, the project is considered ‘protocol approved’ and the project can commence. 

Additional Steps for Third-Party and External Research Surveys

Third-Party Surveys

Third-party surveys are defined here as research instruments (surveys, questionnaires, interviews and others) developed and administered by outside research entities or agencies. They typically, but not always, utilize the agency or entity’s proprietary materials and resources and are solicited to all or a select range of institutions or businesses for a monetary cost to participate.

Often, third-party surveys are preferred over “home-grown” surveys for their ease of administration, benchmarking opportunities, thematic content, or other benefits. Though third-party surveys typically endure a rigorous review and vetting process by the administering agency or entity, review is still requested prior to making an agreement to participate in a third-party survey.

Third party surveys follow the same procedure outlined above. Additional information requested during the proposal review may include:

• Evidence of the third-party’s IRB approval
• Description of what the College must provide to the third-party for administration
• Description of what will be returned to the College (i.e., raw data files, aggregate data, analytical reports, benchmarking reports, etc.)
• Description of any planned incentive program for respondents
• Cost of the survey

Surveys from External Researchers

Reminder: This does not apply to academic research activities, unless those activities are also providing insight into institutional initiatives or decision making.

Surveys conducted by off-campus researchers that require Occidental to provide information about any members of the Occidental community for the research to commence must be done in collaboration with an Occidental College staff or faculty member, who must serve as a primary researcher on the project. Under most circumstances, off-campus researchers who are not partnered with an Occidental staff or faculty member as a PI will not be allowed to survey Occidental community members. In rare cases, off-campus researchers will be allowed to survey Occidental community members without an Occidental collaborator IF they agree to share results in full with a relevant Occidental office or department AND IF said results have the potential to directly benefit that office in its ability to serve its mission to the College. Such projects are requested to adhere to this policy.

Protocols

All surveys intended for distribution to any members of the Occidental community (e.g. students, faculty, staff and/or alumni), by any Occidental student, faculty, staff, department, or program are expected to adhere to the following protocols. These protocols are based on existing policy, IRB standards, and reflect best practices in privacy, safety, and security.

1. All participants must be notified that their participation is voluntary.
2. Respondents must be notified in advance if data collected will not be anonymous.
   1. If subjects are promised anonymity and a login is required to access the survey (e.g., email address, name, or ID number), the researcher must ensure that login information will not be collected and stored in a way that it can be connected to survey results.
3. Personally identifiable information should be collected only as required in relation to the expressly stated purpose of research or a project. This information must be volunteered by the respondent/participant. Arrangements to have information from College data systems attached to the survey data can be requested in the survey proposal.  Approval for requested data elements will be subject to the purpose of the research or project.
4. Survey materials and procedures are unbiased, inoffensive, equitable, and respectful of diversity.
5. Surveys utilize best practices in administration, function, design, and analysis. 
6. Respondents must be protected from risk of unreasonable harm, including any risks regarding confidentiality or privacy.
7. All data collected adheres to College policies regarding information security and privacy ( Appropriate Use Policy ; Information Security Plan ; Electronic Information Privacy Policy ; FERPA ).
8. If your survey or data analysis will include academic, demographic, or contact information for Occidental students, you may be subject to Federal Family Educational Rights and Privacy Act (FERPA) regulations. You are responsible for understanding your responsibility with this information.  
9. For surveys going to a defined list of students where a contact list is needed, the Office of Institutional Research, with assistance and express approval from the Office of the Registrar, will provide the list of student email addresses required for the survey. No other office may provide this list of student email addresses without the consent of the Office of the Registrar. The email list should not be redistributed or used for any other purposes without authorization of the Office of the Registrar.
10. The use of any monitored email listserv to promote or distribute a survey will automatically generate a review.  
11. Students in formal leadership/governance positions are advised not to use the privilege of their positions to send out invitations/links to surveys that have not been reviewed through this process. 
12. Survey data collected on Occidental students, faculty, alumni, and staff surveyed using Occidental resources will be considered the property of Occidental College. The principal investigator must be willing to share the survey data and findings if requested from the Office of Institutional Research or appropriate vice presidents for secondary analysis or to ensure it is maintained as a historical artifact.
13. Use of the college’s survey administration system (Qualtrics) is recommended for conducting campus surveys.  If another system is used, the researcher assumes the full responsibility for the security and privacy of the data.

Certain surveys may require approval by the Institutional Review Board (IRB). The review process outlined in this policy is not a substitute for a review by the IRB. If a survey research project is subject to review by the IRB, approval by both IRB and this policy is required prior to administration.

You may employ the Office of Institutional Research to build and administer your survey for you. If you opt for this assistance, some risks associated with privacy, data security and adherence to College policies may be mitigated.

Resources

Proposal Form

These pieces of information are necessary to complete the review process. Survey content, invite language, and purpose are essential. The time it takes for a survey to complete the protocol varies based on several factors, from stage in the planning process, the scope and scale of the survey, the number of reviews, to the amount of revisions (and subsequent views) needed. The sooner the information is complete, the faster the review process can be. Some information will prompt higher levels of review. Additional questions may arise. ‘Institutional’ surveys (Tier 2) should expect to take longer (typically 1-2 months) than Tier 3 Procedural surveys (typically fully reviewed in less than a week). Tier 3 Advocacy surveys should expect the review process to take 1-4 weeks. While this process is not meant to deny surveys, it may result in a survey being denied, rescheduled, or re-envisioned. Feedback on the rationale for the decision will be shared.

When you are ready, you may access the proposal form using this link: IR Survey Proposal Form. 

• Requestor Info: Name, Affiliation (student, faculty, staff, etc)
• Primary Survey Sponsor: (could be a committee, group, department, or individual)
• What other individuals, areas, departments are sponsoring, or have you consulted with on this survey project? 

• What does the survey [need to] ask about? (content)
• What will the data be used for? (purpose)
• Who is the survey going to?
• How is the survey going to be distributed?
• When is this survey going to be administered?
• What personal or identifying information will be collected?
• How will anonymity or confidentiality be ensured? How will the data be stored?

• How long is the survey (# of responses a respondent must make)?
• Will this survey need to be repeated?
• Will an incentive be offered?
• Is it administered by a third-party?

• What is the recruitment/invite language?
• What is the plan for analyzing the data?

Software

Occidental College has an institutional subscription to Qualtrics University Research Suite (http://oxy.qualtrics.com/). It is encouraged to use this preferred vendor for survey projects. Follow these instructions to set up an account. Qualtrics provides training in various forms on their website. 

Appendix 

Occidental Survey Policy: Basic Step-by-Step Guide

Familiarize yourself with the policy, and procedures and protocols therein. Consider if your project should fall under the policy. IR can help make this determination. Refer to the categorization of survey activities and decision flow documents for reference. Submit the project as a proposal to IR (in-person meeting, email, online form). Include as many pieces from the proposal form as possible. IR will review the proposal within 3 business days and communicate next steps. These may include (as appropriate): 

1. Approval to administer the survey as is. 
2. Project is being reviewed as is at a higher level. 
3. Follow-up or clarifying questions about the project or materials. (this may pause a review at any level until resolved)
4. Suggested edits, revisions, changes. (this will pause a review at any level until resolved)
5. Suggested collaborations. (this will pause a review at any level until explored) 
6. Schedule a meeting to develop the project further.

Once all levels of review at proposal and final stage are satisfied, the project is deemed ‘protocol approved’. 

Tiers: Categorization of Survey Activity

	Purpose:	Characteristics:
Tier 0: Not Subject to Policy	Surveys for academic research; Course evaluations Recruitment forms, applications, or invites	Academic research Course evaluations Faculty research Funded research Course-related research Instruction/teaching evaluations Solicits contact info, etc from interested parties only Purpose is to recruit respondent for volunteer, service, membership, employment, participation or other activity.
Tier 1: 'Self-Assessment' Surveys	Assess a department, group, program, event under my immediate ‘control’.	Involves a participant group, roster, membership list, attendee list, or specific group feed Could be scheduled, as needed, or point-of-service Topical or specific to the entity being assessed Used to address practices, operations within the survey sponsor’s control.
Tier 2: Pulse Surveys	To collect information to give to decision-makers or inform decision-making on College policy, practices or operations.	Could go to any target population. As needed and/or timely Topical or specific with few items (typically) Procedural surveys collect preferential choices based on best ‘fit’, available options, or characteristics of a task, activity, practice, procedure, or operation.  Advocacy surveys strive to identify shared perceptions, opinions, experiences, expectations or ‘voices’ that advocate for changes to policy, procedure, practice or operations.
Tier 3: Institutional Surveys	Support institutional priorities or requirements	Goes to a population or sub-population Scheduled (on the calendar) Covers multiple themes with many items linked to priorities   -OR- Fulfills obligations or requirements of external governing agencies.

Decision Flow: Does my survey require review under this policy?

The flow chart provides guidance on how your survey projects fits under this policy. 

Path from 'No' above: 

Levels of Review

Level:	Role:	Surveys Reviewed:
A - Email list moderators; Contact providers	Approve surveys that fall under Tier 1 for posting; route surveys needed additional review to other levels.	Surveys posted to moderated email lists or sent to contact lists
B - Institutional Research	Assist in determining if ‘non-institutional’ applies; reviews survey needs to determine strategy; identifies alternative info sources; reviews proposed methods and designs for survey projects; connects with collaborators and coordinators; determines if and which additional levels of review are required; facilitates next steps.	All surveys
C - Stakeholders	Advises on content, topic, need, and value of information; suggests collaborations and alternative info sources.	All Tier 2 Advocacy surveys; Some Tier 3 surveys
D - Senior Leadership	Advises on institutional need and value; determines courses of action.	Some Tier 2 Advocacy surveys; All Tier 3 surveys

 

What might prompt a review at a particular level?

• Topic - If the topic is related to institutional priorities, impacts more than one area of the college, or is of a sensitive or polarizing nature, it will require more or higher levels of review. 
• Purpose - If the purpose is to advocate for change, all areas impacted will be consulted. 
• Group being surveyed - Surveys going to the entire college should be reviewed by a member of senior staff. The survey population may spark consultations with College partners connected with that group. 
• Survey sponsorship - A survey sponsored by a Vice President will not require as many levels of review, whereas a survey sponsored only by a student group/organization will require more levels of review. 
• Survey administration - A survey administered through a moderated email listserv will be reviewed starting with the moderator.