Professor Frank Sommer, 42, a Berlin-based urologist, said that the substance could just tip the balance.
"A substance like that could shift the balance of the sex hormones in men towards oestrogen," he said.
"In the long term this leads to less sexual drive, encourages the belly instead of the muscles to grow and has a bad effect on erection and potency."
BPA has already been banned in Canada and three US states.
Bottles and cans containing the chemical have been linked to breast cancer, heart disease, obesity, hyperactivity and other disorders.
Most manufacturers of baby bottles have stopped putting it in their products but older stock containing the chemical is still on sale.
The US Food and Drug Administration (FDA) supports its removal and has stated concerns regarding the impact of the chemical on babies and young children.
BPA is known as an endocrine disrupter and interferes with the release of hormones.
It can affect disorders associated with metabolism, fertility and neural development.
It is widely used in tins of food and canned drinks to toughen the internal lining of the container.
In December last year seven experts from five British universities wrote to Andy Burnham, the health secretary, calling for a review of BPA.