Ölgii has many modern statues besides its ancient stone men. The central square is dominated by the Soviet-Mongolian War Memorial commemorating Mongolia's contribution to the Great Patriotic War (World War II). Facing the square is statues of Lenin and Sukhbaatar ('Ax Hero,' hero of the war of independence) in front of the government building. Inside the beer garden is a full size statue of a man playing a dombra (traditional Kazakh instrument). The most unusual statue is a full size statue of a local war hero, Akie, that died in 1929 during the war between White and Red Russians in Mongolia. Akie is firing a pistol backwards while running.
Myзeй
You should take time to visit it before heading out to the countryside to see the displays of Kazakh culture, local history, and wildlife. The 3-story museum has an entire floor dedicated to various tribes living in Bayan-Ölgii including the majority Kazakhs, Uriankhai, Dörvöd, Tuva, and Khoshuud, including a complete Kazakh ger (much larger and more colorful than Mongolian gers). There are also costumes and ancient artifacts from each tribe. The second floor has local history from the Communist era, and the ground level covers local wildlife, with many stuffed animals, and has a gift store full of Kazakh handicrafts. Outside is a small display of ancient small Turkic Stone Men (ones in countryside are much bigger).
It is a nice place to hang out in the evenings; has cheap beer and soft drinks, and is one of the greener spots in town. Most tourists come in the summer when it is quite brown and dusty (in the winter it's snowy!), so green is a rare thing.
This giant red building was constructed to promote Kazakh culture and host events in the city and now has concerts and plays from locals and groups traveling from Kazakhstan or elsewhere in Mongolia. Though it is not especially nice or well maintained, it does hold regular concerts and plays throughout the year.
It's worth going on hikes up into the nearby mountains; find someone familiar who can guide you there, or just walk in any directions. Bugan Tou Mountain to the southwest can be hiked at 2775 m.