Photo: そらみみ, CC BY-SA 4.0 via Wikimedia Commons
This area has several historical shrines and Buddhist temples, including:
© OpenStreetMap contributors
博多町屋ふるさと館
gives you a glimpse of the Hakata way of life.
The 8th-century Kushida Shrine is starting point for the annual Gion-Yamakasa Festival.
Japan's first Zen temple.
Hakata-za
An experience worth checking out. Check times and prices at the Hakata-za near Nakasu. If you don't want to stay for the whole show, or don't have so much money to spend, you can watch part of a show for about ¥800; see here for details. Half price student/young (~25) adult tickets may be available day of performance to be released 20 minutes prior to the show; see here for details.
This gallery hosts temporary exhibitions from Kyushu and all around Asia. It has a good shop.