Baracoa's old town, which pretty much encompasses the entire city, is not particularly pretty nor does it contain many particularly notable buildings, but it's pleasant to stroll in.
Effectively the city's main plaza, Parque Independencia features a fountain, a statue of local hero Hatuey, and Baracoa's original church (closed for renovations as of 2012). Most of the tour operators' offices, including Cubatur's, ring the park, as do the city government offices. Many casas and paladares are nearby.
Baracoa has its very own Malecòn, running from the northern bus station to Fuerte Matachín, though it has none of the fame (or crowds) of Havana's. It does, however, make for a pleasant stroll along the Atlantic. Near the middle of the walk, a park and statue commemorate Columbus and the Spanish landing on Cuba. Some casas are along the Malecòn, offering great views onto the sea.
Hotel El Castillo
The highest fort in Baracoa, dating to 1739, is now a hotel, but non-guests are free to wander the outdoor (and some indoor) areas and admire the views.
More a historical oddity than a great place to spend the night, this hotel along the malecón was run by a Russian woman who escaped to Cuba from the Russian Revolution – only to become one of Castro's closer confidantes.
At the southern end of the Malecòn this former Spanish fort now serves as the city's municipal museum. It's quite informative, but most exhibits are in Spanish.
A place to learn about the cacao plantation, production and history in Cuba. A few 100 m further there is a river where you can take a dip.