The fire broke out around 6:30 am at the former Merced County library building on 22nd near M Street.
"Tragic ending for a majestic building," said Merced County Sheriff Vern Warnke.
The Merced County Historical Society says the structure was originally the county's first high school, which was built in 1897.
Sarah Lim, the director of the Merced County Courthouse Museum, says by 1920, students outgrew the school.
"After that it was renovated and remodeled into a county library, and it was used from 1920 to 1975," said Lim.
Since then she said the desolate building sat vacant, but was still filled with memories.
"It's one of the oldest buildings in the city of Merced, and its listed in the national registry of historic places," added Lim.
Through the day and into the evening, crews fought the flames and tried to protect nearby buildings.
Despite the damage to the Romanesque-style structure, the building was still standing hours after the fire started because of its unique architecture.
"It was built with red bricks, so when you look at original photos of the building, you see red bricks all over, and over the year's they put plaster all over the building," Lim explained.
Eventually, a demolition crew was brought in to help tear down the building.
Historic building goes up in flames in Merced
"We needed to bring a demolition crew in in order to take down some of the walls cause they were posing a collapse hazard," said Merced City Fire Battalion Chief Tony Giotta.
"We couldn't have fire fighters working inside the building with the walls still standing in the condition that they were in."
Local leaders say before the fire, there were hopes of restoring the three-structure building but the price tag for the potential project prevented that from happening.
"If we would have had the financing way back in the day to rebuild it or just make it usable again so that our citizens could use it," Sheriff Warnke said.
Historic building goes up in flames in Merced
"There was always a potential but everything cost money. To have it like that and to have it end tragically this way, it's sad for the county."
There are concerns if the walls of the historic building collapse, it could impact the superior court building next door.
The fire department says it's expected the court building won't reopen until Wednesday.
The cause of the fire is still under investigation.
For news updates, follow Brianna Willis on Facebook, Twitter and Instagram.
For news updates, follow Kate Nemarich on Facebook, Twitter and Instagram.