Henry Langen of Pembrokeshire Access Group (PAG) also said that many were not leading to prosecutions.
The Crown Prosecution Service (CPS) and Dyfed-Powys Police will co-host the day-long event at Narberth, with PAG.
Among the speakers will be Simon Green, 34, of Bridgend, a wheelchair user who has been a victim of hate crime.
The conference, at Hotel Plas Hyfryd, Narberth, is open to any disabled persons who have
Mr Green, who has been a wheelchair user for six years, said during that time he had been physically and verbally assaulted because he is disabled.
For Week In, Week Out he secretly filmed a couple of his nights out to expose the hostility and abuse he sometimes experiences.
During one evening he was confronted by a group of men who verbally abused him, swore at him, called him a "cripple", and suggested he could really walk.
Mr Langen said: "There's not enough of these cases reported to the police and with regard to the ones that are reported there are not enough prosecutions.
"The CPS and the police need to have a rethink on this."
A spokeswoman for the CPS said it wanted to encourage more people to come forward and report any instances of disability hate crimes.
All reports would be treated seriously, the spokeswoman added.
"We are starting to look more closely at disability hate crime and improve what we are doing with it," she said.
"There are still improvements that can be made."
BBC News
