There are some nice hikes right around Cayenne. One of the easiest to access is the Sentier de Montabo, which connects Cayenne proper with the nearest beach, and on the way gives you close-up views of both of French Guiana's two major native ecosystem types: the mangrove swamp and the Guianan tropical rainforest.
A park with high palm trees, lively in the nighttime.
Remains of a historic fort on a hilltop right on the west side of downtown Cayenne. Great views, and probably the best (only?) place to see the ocean and Cayenne River from downtown, as well as an overview of the city itself. Well lit into the evening.
Cayenne cathedral
Cayenne's cathedral is a major landmark, the seat of the Diocese of Cayenne and a National Monument of France.
A 19th-century fort, nowadays a Monument Historique (French National Heritage Site).