Extensive pilot study conducted by JNLR reveals that existing methodology is still best measure of Irish radio listening habits

A new pilot study conducted by JNLR, Ireland’s national listenership survey, has found that’ Day-After Aided Recall’ remains the best methodology measure of Irish radio listening habits.

The role of the JNLR Committee is to ensure the radio industry in Ireland and its stakeholders have access to the most robust and reliable listenership data available, carrying out a national sample of 16,500 respondents annually.

The JNLR Committee undertook a review of the approach currently adopted in the Irish market, comparing and contrasting it with alternative methodologies used in other markets, with a particular focus on the approaches being used and under consideration for the UK market.

Currently there are three broad philosophies for radio research measurement worldwide:

  • Day-After Aided Recall – what was listened to the previous day
  • Diary self-completion – where a respondent records their radio listening in a diary over seven days
  • Passive measurement – where a device, carried by the respondent, records the radio stations that the device/respondent could hear.

The current JNLR uses Day-After Aided Recall, interviewing people in their own homes and asking about radio listening on the previous day. This approach has been adopted since 1989 and has provided the industry with a remarkable level of data reliability.

In the UK, a Diary Self-Completion approach is used, and the JNLR Committee study involved trialling this approach in Ireland to find out if the data delivered was of a higher quality than the Day-After Aided Recall approach.

For the Diary Self-Completion trial, a sample of 650 respondents were recruited across the Dublin and South East (Carlow, Kilkenny, Wexford, Waterford and Tipperary SR) regions to match the profile of sample required for the JNLR. Similar to the standard JNLR survey, a range of quota controls were applied in terms of age, gender, social and working class. Each recruit was given the choice of completing a pen-and-paper or an online diary.

Participants were asked to complete the diary every day for seven days, although due to the self-completion nature of the task it is not possible to say exactly how much time did elapse between listening and completing the diary.

The diary trial findings revealed no significant data differences between the pen-and-paper self -completion diary and the current JNLR. For the online self-completion diary, listenership levels were unusually low, suggesting poor levels of compliance, possibly driven by the inconvenience of having to go online and log in each time a respondent needed to record a listening event. Overwhelmingly, even among younger respondents, the pen-and-paper self-completion diary was preferred over the online diary as the more convenient option.

The study concluded that a diary approach (pen and paper or online) does not provide better data, may impact data continuity and may require a move to a smaller sample size to reflect the increased difficulty in recruiting and monitoring diary respondents.

The third option open to the radio industry is to passively measure listenership by maintaining a panel of respondents who are required to carry a listening device at all times that would record the radio stations listened to. Over the past decade, the JNLR Committee has been closely following developments in passive measurement, with particular attention being paid to developments in the UK. On a number of occasions, international experts have been invited to present on latest developments in passive measurement to the Committee.

Passive measurement remains cost prohibitive for a market as rural and geographically distributed as Ireland. While the cost of the devices used has fallen (with the use of a smartphone, to ‘hear’ now a possibility), the costs of recruiting and maintaining a panel, and the costs associated with maintaining an audio-matching or encoding infrastructure, have not decreased.

The JNLR study concluded that, while cost is a key barrier, it is not the only barrier, as a move to passive measurement would also impact data continuity and there is no guarantee that passive would produce more accurate data: the technology measures the radio we can ‘hear’ and not the radio we ‘listen’ to. Additionally, passive technology relies on achieving a high level of compliance from panellists.

Having undertaken a full review and explored international best practice in radio audience measurement, the JNLR study concluded that the current Day-After Aided Recall methodology remains best practice for the Irish market at this current time.

However, while Day-After Aided Recall is the best approach for radio in Ireland currently, new approaches to data collection will form part of the JNLR Research Tender 2020 – 2022, for which expressions of interest are currently being sought.

Scott Williams, Independent Chair of the JNLR Committee, said:

“This study is a very satisfactory endorsement of the robust nature of the existing JNLR methodology. While there are a number of alternative methodologies for recording radio listenership currently in use, it’s clear that Day-After Aided Recall remains the gold standard for the Irish market. We would like to thank all the members of the JNLR Committee for their time in conducting this review and we are delighted that all members of the Committee, including members who represent advertising agencies (IAPI), advertisers (AAI) and the regulator (BAI), fully endorse its conclusions. It has always been important for radio in Ireland that the industry and all its stakeholders speak with one voice and work in partnership to promote the wonderful medium of radio.”

The JNLR Committee members are:

  • Scott Williams, Wireless (Independent Chair)
  • Barry Dooley, AAI
  • Andrew Robinson, BAI
  • Helen O’Rourke, IAPI (agency representative on IAPI Board)
  • Katie Boylan, IRS+
  • Gavin Deans, Media Central
  • Diarmuid O Leary, Red FM
  • Gareth Ivory, RTE
  • Dan Healy, RTE
  • Keith McCormack, Today FM / Communicorp
  • Brian McCarthy, Urban Media