The ABI's latest report revealed in 2013 there were 353,000 annuities sold by its members in the UK, worth £11.9bn in total.
Members have been reporting that the number of customers deferring has risen.
Hargreaves Lansdown said the fall in annuity sales is the opposite of what it expected based on demographic data.
{desktop}{/desktop}{mobile}{/mobile}
The figures have been released hot on the heels of the FCA's annuities review, which concluded the market is failing consumers.
The FCA report, published on Friday, found most retirees are missing out on significantly better incomes,  with a failure to shop around highlighted as a key reason.
The ABI data shows that shopping around was worst amongst those with the smallest pension pots – with only 46% of customers with a pot of less than £10,000 saying they shopped around compared to 63% of the total.
Otto Thoresen, ABI director general, said: "We agree with the FCA that people with small pots need help and flexibility to make the most of their savings, and increase their options at retirement."
Tom McPhail, Hargreaves Lansdown's head of pensions research, said: "The retirement income market appears to be shifting, with demand for annuities collapsing and surging interest in drawdown."
Hargreaves' figures from 2013 showed demand for drawdown increased 40%.
Mr McPhail said: "The first wave of baby boomers has passed age 65 but even so, we would expect annuity sales to stay high yet they were lower in 2013 than in 2011, when fewer people were reaching their mid-sixties.
"There are lots of factors at work here, including the wider economic background, sentiment about future interest rates and actual retirement ages; low annuity rates in recent years compared to historic averages may have had an impact too."
Other key figures from the report were that the proportion of annuities bought on the open market has increased in recent years.
Some 48% of annuity sales were external in 2012 and 2013, compared to 31% in 2003.
The proportion of medically underwritten annuities continues to rise and was at 28% in quarter four in 2013.