Support #1007

Is there a gap in the data, in the year 2009?

Added by Luis Valenzuela about 2 years ago. Updated almost 2 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:



I am reading your documentation about the Harmonised US-BHPS data (, and I cannot figure out whether there is a gap in the BHPS data in 2009. From Figure 1 in the main documentation pdf I see there is such sampling gap, but nowhere in the text I can read an explicit mention that there might be a data gap (i.e. that all questions in 2009 are missing). If I want to use a long panel between 1991 and 2016, is there a gap in 2009? I have looked at US documentation and I don't find this mentioned either. Given that such data gap would be very important and is not mentioned everywhere, I get the impression that there is information for households and individuals in 2009 (income, etc). Can you please let me know which is the case? I think that making this very explicit in the documentation would be very handy for researchers considering working with BHPS-US data, before actually embarking in such project.



#1 Updated by Stephanie Auty about 2 years ago

  • Category changed from Data linkage and consents to Harmonisation
  • Status changed from New to In Progress
  • % Done changed from 0 to 10
  • Private changed from Yes to No

Many thanks for your enquiry. The Understanding Society team is looking into it and we will get back to you as soon as we can.

Best wishes,
Stephanie Auty - Understanding Society User Support Officer

#2 Updated by Stephanie Auty about 2 years ago

  • Assignee changed from Stephanie Auty to Luis Valenzuela
  • % Done changed from 10 to 70

Dear Luis,

BHPS ended in 2008 and the last interview was the 18th wave of BHPS. The interviews were conducted mostly from Sep-Dec 2008 with some cases spilling over into the next few months. The BHPS sample members were eligible for interview again in 2010 as part of the second wave of Understanding Society: most of the BHPS sample members were interviewed from Jan-Dec 2010 and a interviews few spilled over into the next few months. So, depending on the BHPS Wave 18 interview date and the UKHLS Wave 2 interview date, the gap between these two interviews can range from 10 to 30 months. In other words, there is a data collection gap (as shown by the dark blue bars in Table 1 of the Waves 1-7 User Guide and in Figure 1 of the BHPS-Harmonised User Guide), but I'm not sure what you mean by a sampling gap.

You can read more about this issue in section 3.5 of the Mainstage User Guide and in the BHPS-Harmonised User Guide, both of which are available to download here:

Best wishes,
Stephanie Auty - Understanding Society User Support Officer

#3 Updated by Luis Valenzuela about 2 years ago

Thanks Stephanie.

So, to put it differently, whereas in BHPS waves pretty much align with years (e.g. in a panel data study one could replace wave with year), this is no longer the case in US, as waves actually run for two years. So, to assign years from US into a panel data analysis one requires to use information from "istrtdaty" and "istrtdatm" variables in order to assign interviews to BHPS-type waves. In this sense, there is a sampling "dip" in the panel-data structure for those individuals coming from BHPS into US in 2009, because BHPS interview dates were mainly in Sept-April, whereas BHPS "2009 cohort" was interviewed in Jan-Dic 2010, and thus, some of them would belong to the "2010 cohort" in BHPS.

This clarifies my issue. However, as far as a I can see, there is no mention of this in the documentation. And yet, this is crucial for anyone using panel data analysis. For instance, if one wants to add a time trend, or use year-dummies, using waves as years is misleading, because within waves the response of an individual (e.g. wages) refer to June of year X whereas for another individual in the same wave it corresponds to June of year X+1. This also means that longitudinal weights are not particularly helpful either. Actually, given the new US structure, they are probably never helpful. The only solution is to compute them not based on the variable "wave" but on a manually derived "year" variable, which currently is not available. This is probably something to be considered for future releases, as the key purpose of US - panel-data analysis - is in my opinion severely hampered with the current interview/wave structure.


#4 Updated by Stephanie Auty almost 2 years ago

Dear Luis,

This is not a sampling dip as the sample is not drawn each year, it is just a longer time between interviews than between other waves. BHPS Wave 18 interviews took place between September and December 2008 so it would be too soon to interview the same participants again starting in January 2009. Figure 1 of the BHPS-Harmonised User Guide clearly shows that they were not interviewed in 2009.

This working paper describes the sample design of Understanding Society and how the BHPS continuing sample members were incorporated into year 1 of Wave 2:

There is a guide to selecting weights in the User Guide, in section 3.9.1, including specific weights for using the BHPS sample members. If you have further questions on which weight to use I can refer this issue to the weighting team.

Best wishes,
Stephanie Auty - Understanding Society User Support Officer

#5 Updated by Stephanie Auty almost 2 years ago

  • Status changed from In Progress to Feedback

#6 Updated by Luis Valenzuela almost 2 years ago

Thank you Stephanie, that guide is very helpful. For the moment I've resolved the issue, so do not need further help with weights.


#7 Updated by Stephanie Auty almost 2 years ago

  • Status changed from Feedback to Resolved

#8 Updated by Stephanie Auty almost 2 years ago

  • % Done changed from 70 to 100

Also available in: Atom PDF