The biggest threat within technology is of course security and that of the privacy of those of us that use. With the smart card, that is also a concern. Because it is not as wide spread in the world yet - despite being very popular in Europe - there are issues with things not matching up. Magnetic stripes are still used for credit cards, either still issued with smart cards as well, in order to standardize the technology. There is also the issue that while the cards may be able to be secure, it does not mean that the machines that they use will be.
There is also the public perception of privacy. If the public doesn't believe something holds to a high level of security, they of course won't use it. There is also the Orwellian thought of "Big Brother" watching our every move as technology becomes more and more ingrained within our daily life styles. The very concept of the card is to allow smart card readers 'read' the information, again allowing for quick access to certain information, just as bank or credit card amounts in order to pay for groceries at the store.
But there lies the problem that with that information gathered, anyone could have access to the information stored. Along with this, what should be the aftermath if a memory problem causes the smart card to erase information from itself? For example, if your smart credit card's memory suddenly goes to a default status and reads no money within the bank account or on the card, who should be responsible for this? The user, who may have accidentally jumped in the pool with the card on him? Or the bank, who were unaware that the credit could be affected in such a way?
Also, while smart cards themselves are relatively cheap to produce and market, the readers themselves are sometimes costly for a business to install, making the usage that much smaller as they are not enough avenues in which to utilize the smart card.