But he said Greece was worthy of its place because it "abides by the rules" and was "not the only eurozone country with a deficit of that order".
His comments come ahead of the Greek government outlining its plan to deal with the country's debt.
Mr Papaconstantinou stated: "Greece is not the next Iceland, nor is it the next Dubai... it is tackling the very serious situation that we have.
"It is doing it with specific measures that cut expenditure and increase government revenue and also it is a government which is immediately tackling long-term structural problems," he added.
Last week, international ratings agency Fitch downgraded Greece's credit rating - meaning that it thought the country was now a riskier place to invest.
Since then Greece has come under increasing pressure to take action over its deficit from the European Central Bank.
However, Collin Ellis, European economist at Daiwa Securities, says that "the idea that the euro area is on the the brink of losing Greece and possibly other members is simply absurd".
He argues that there is still time for the Greek government to sort out its finances, and, should it not be able to, "it is inconceivable that other euro area member states and, if necessary, international organisations would not step in".