Yes, there can be downsides. Depending on the type of lens adapter, you might lose certain functionalities such as auto-focus or auto-exposure. Additionally, using an adapter may introduce slight degradation in image quality due to the added distance between the lens and sensor, or may lead to slower focusing times.