The EDL emerged last year and has held demonstrations in a number of towns and cities against radicalisation.
But the West Midlands Counter Terrorism Unit has told BBC Radio 5 live there is evidence EDL events can encourage extremists.
Officers also say they are worried about radicalisation inside prisons.
The BBC was given exclusive access to the West Midlands Counter Terrorism Unit - housed in one of the most secretive buildings in the UK.
Thousands of people will have walked or driven past the anonymous offices, somewhere in the West Midlands, without realising that inside lies a counter-terrorism hub.
The only clue is close-up where security is tighter than for neighbouring buildings. But there are no armed guards, barbed wire or guard dogs and, of course, there are no signs.
Det Ch Insp Alex Murray walks me through the gate and into the building.
"There's always a certain element within the violent extremist community that may want to target a premise like this," he says. "So for that reason it needs to remain discreet."
It took months of delicate negotiations before I was allowed in, and it only happened under the condition that I did not reveal its whereabouts.
'Covert environment'
The intelligence community is beginning to believe that it is in its interests to be more open.
Despite this, Det Ch Insp Murray admitted that people were nervous about my visit, saying: "Historically, and for very good reasons, it's been a covert environment. People don't want to become targets themselves."
Some areas were strictly off limits. Next to a door marked with a sign that indicated top security clearance was needed, I was told that "in theory" I might be allowed in, but I would need to be vetted, specially trained, and probably would not be allowed to reveal what I saw.
A lot of the building was accessible though. There is a large open dining area, with kettle, microwave and vending machine. Nearby there are changing rooms and a small gym. Upstairs there's office space where more mundane work goes on and a large meeting room.
In the centre of its conference table were two phones with ultra secure lines.
One, marked Tacit Secret, can only be used in this country. The other is activated by a special key and can contact similar agencies around the world.
'Classified info'
The West Midlands Counter Terrorism Unit was set up six years ago, the first outside London.
Its boss is Det Ch Supt Matt Sawers. He says: "We put together a small group of about 80 people, but some of the threats that started to present themselves meant that we needed to build up the numbers."
I asked how many people currently work for the West Midlands CTU, but was told that the answer is classified.
The officers talk about current threats from what they call the al-Qaeda "franchise" and how links to Somalia and Yemen are giving them as much concern as more "traditional" areas like Pakistan and Ireland.
They are worried about radicalisation inside prisons as well as colleges and universities, and they also fear that groups with an anti-Islamic message, particularly the English Defence League, are acting as a recruiting sergeant for Muslim extremists.
Many EDL demonstrations and counter-demonstrations have ended in violence, and Det Supt John Larkin tells me that they have witnessed signs of radicalisation afterwards.
"They look for the hook to pull people through and when the EDL have been and done what they've done - perversely they leave that behind," he explains.
Detection strategy
Inside the brightly-lit modern building, staff with police, military and intelligence backgrounds work in two broad areas.
The first is detection. These are the officers who respond to any incidents or threats. Operations are either intelligence-led or in response to tip-offs from the public.
"There is normally a steady flow of information. You can quickly see which are the calls that will lead to immediate action," says Det Insp Darren Walsh, who is in charge of the Initial Response Team.
West Midlands CTU's biggest success so far came during Operation Gamble in 2006, when a plot to kidnap and behead a serving British Muslim soldier was foiled.
Its officers can be deployed anywhere and the unit's forensic team spent many weeks in London after the 7/7 attacks.
Besides detection, the next area is prevention. This unit aims to build strong links with communities to try to stop radicalisation taking place.
After considerable success, there has been a major setback. This summer, West Midlands Police became embroiled in a scandal over surveillance cameras which were erected in predominantly Muslim areas in Birmingham.
Residents were told they were part of a crime-fighting initiative, but it emerged that they had been paid for out of counter-terrorism funds.
Everyone from the chief constable down has said sorry and the cameras are due to be taken down. But it has affected relationships and CTU knows they need to be rebuilt.
The government has also announced a national review of its Prevent Strategy, which is expected to report in the new year.
BBC News
