Notes From a Fellow: Three Skills to Sharpen for a Career in Applied Psychological Science

If you’ve read previous installments of this column—and I hope you have!—then you know that I’ve been sharing my reflections during my Fellowship at the Office of Evaluation Sciences (OES) in the U.S. General Services Administration. After nearly a year of experience, I have a better understanding of the knowledge and skills used at OES that are not necessarily covered in a typical psychology graduate program. For those looking to take on a similar role, here are my suggestions for skills you might want to develop.  

Be able to articulate and apply psychological science in the design of interdisciplinary applied interventions.  

Understanding applied intervention design, being able to do field experiments, and working with administrative data are my “Big 3” skills to develop, but different roles within the government might require other skills from psychological scientists.

Our team at OES is a mix of policy-area specialists (with expertise in housing or education, for example), methods specialists, and “all-rounders.” (Read about some of our different perspectives.) I consider myself an all-rounder because I work with psychological theories of motivation that are relevant to behavior in many policy areas. Most people with a graduate education in the psychological sciences could play a similar role even if, like me, they don’t have any particular public policy expertise. 

However, the way interventions are developed in an interdisciplinary applied setting like OES is probably different from how it’s done in most graduate schools. One tool OES uses is a “map” that shows all the steps to interacting with a program, as well as the psychological and structural barriers that individuals might encounter.  

These maps are similar to the “customer journeys” used in marketing. However, customer journeys often result from qualitative research with a small number of participants. OES sometimes uses qualitative research, but our maps typically synthesize relevant quantitative research from psychology and related disciplines. These maps make it easier to identify the barriers that can prevent people from executing a specific behavior—for example, factors that discourage government employees from making earlier, and cheaper, bookings for work-related travel—and draw on psychological science to identify promising interventions that will address those barriers (for a related OES map). 

Maps aren’t a necessary part of applied intervention design, but they are one way to home in on relevant research and theories. These maps can also be a great way to facilitate conversations among people with different disciplinary backgrounds.  

Understand special considerations that pertain to field experiments. 

Large field experiments require at least two specific skills that many psychology graduate programs don’t cover in much depth. The first is block randomization, a technique I’d read about but never considered using until I joined OES. (Fortunately, OES has methods specialists, as mentioned above, to guide the way.)   

Blocking is a tool for refining random assignment. It involves creating homogeneous subsets, or blocks, of experimental units (e.g., research participants) and randomly assigning treatments within those blocks. Specifically, preexisting variables that could affect outcomes of interest are evenly distributed across the blocks, which allows researchers to make more precise estimates of an experimental treatment’s effects.   

For example, a current OES project is evaluating a multimodal communication strategy to increase take-up of the American Opportunity Tax Credit among students at a Midwestern U.S. university. Blocking was based on student characteristics such as years in school, status as a transfer student, and status as a dependent on someone else’s tax return. These characteristics probably predict differences in tax filings, so if we ensure that they’re evenly distributed across the groups that do and don’t receive targeted communications, we can be more certain that observed effects are due to the communication strategy.   

In addition to specific skills like block randomization, large field experiments require project management skills, because the contributions of multiple people must be organized and tracked over time. By seeking out opportunities in graduate school or your current position, you may be able to develop general project management skills as well as the ability to use specific tools like block randomization. 

Gain comfort working with administrative data.  

I said a bit about this already in my March/April column. The specific challenges of using administrative data include identifying relevant existing data sets, accessing and cleaning the data, and leveraging them to build evidence. Also, although using administrative data is typically less expensive and less burdensome than collecting new data, it can limit the information you’re able to observe. 

It seems like data science skills will be increasingly relevant to the work OES and similar teams do. For example, some current OES projects are examining how funding to provide relief to small businesses during the COVID-19 pandemic affected those businesses’ resilience in San Diego and Dallas. The analyses combine administrative data about which businesses applied for and received funding (from the cities) with data on bankruptcy (from court records), opening hours (from Yelp), and credit card transactions (from card payment databases).  

Having team members who can find these different data sets and wrangle them into usable form allows OES to do evaluations that would be impossible otherwise. These projects are different from typical OES work, in which government agency collaborators usually have their own data. But in almost every kind of project, we need to brainstorm creative measures of the core outcomes of interest and then be able to work with the data an agency collaborator provides.   

Understanding applied intervention design, being able to do field experiments, and working with administrative data are my “Big 3” skills to develop, but different roles within the government might require other skills from psychological scientists. As graduate programs look to prepare their students for these sorts of jobs, I’m hopeful that they’ll offer coursework and other opportunities to build the necessary skills. If you have examples or ideas to share, I’d love to hear about them (via Twitter @heatherkappes or by email at [email protected]).

Feedback on this article? Email [email protected] or scroll down to comment.