The charity Keep Wales Tidy is among those calling on people to recycle rubbish to protect Cardiff Bay.
The barrage holds water against the oncoming tide, but large amounts of plastics and other general waste is being thrown in and building up.
“People like to pass the blame when it comes to recycling, but it actually starts with individual responsibility,” said Keep Wales Tidy spokeswoman Nia Lloyd.
“Almost 80% of marine litter comes from land bases,” she added. “People need to change their ways to stop this rubbish going into the sea in the first place.”
The charity said plastic collecting in corners of the barrage also posed a danger to wildlife.
Cardiff Bay is a popular tourist attraction, but for visitors and residents, the state of the water is a concern.
Nicola Johns who lives near the bay, said the sight of the rubbish made her ashamed to live there.
“We were here celebrating a birthday with family and friends last Sunday, we were walking around and it was embarrassing,” she said.
“The bay was packed and all you could hear was people talking about the amount of rubbish in the water.”
Mrs. Johns, who enjoys photographing the bay said the rubbish problem has become so bad she’s had to edit her pictures to cut out the plastic bags, bottles and even footballs floating in the water.
Another Cardiff resident Caroline James called on the council to take action, saying: “Somebody here needs to come and pick up the rubbish, there’s enough bins around here.”
In a statement, Cardiff Council said: “Keeping the water clean is an ongoing task and we would urge people visiting the Bay to dispose of their waste responsibly to help us achieve this.”