Yhyjyk

There was an anime that I forgot a long time ago, basically there were these legendary swords made from talismans and this dude could change them, and his body was shattered into bits so he almost dies whenever he uses one of them. The reason his body is shattered is because he got into a huge battle with like a demon or something and that was the price to put it back to sleep or kill it idk. I think he was like a military general and he was sent to an orphanage to run things there..

I believe the anime you are after is commonly called World End, it's full name in English is What Do You Do at the End of the World? Are You Busy? Will You Save Us? and in Japanese 終末なにしてますか？　忙しいですか？　救ってもらっていいですか？ Shūmatsu Nani Shitemasu ka? Isogashii Desu ka? Sukutte Moratte Ii Desu ka?

This anime starts with a guy who is looking for work and ends up getting a job (through a lizard-man(?) friend) working for the military

looking after some secret weapons, which just happen to be little girls/fairies.

As the story progresses and we find out more about the character we come to know he has been frozen for 500 years after battling a demon and as a result of his revival is very weak and cannot fight like he used to. That being said, he still has the ability to get the maximum power out of the swords that they use which happen to be made up of fragments as you said.

For more info look https://myanimelist.net/anime/33502/Shuumatsu_Nani_Shitemasu_ka_Isogashii_Desu_ka_Sukutte_Moratte_Ii_Desu_ka
or
wikipedia
.

With Theyna's hint, I found Sukasuka, published in English as WorldEnd.

It's been over 500 years since the human race almost went extinct at the hands of the fearsome and mysterious "Beasts". The surviving races now make their homes, towns, and cities up on floating islands in the sky to keep out of reach of all but the most mobile of Beasts. Only a group of young girls, dubbed the Leprechauns, can wield the ancient Dug Weapons needed to fend off invasions from those dangerous creatures. Into the peoples' unstable and fleeting lives, where a simple call to certain death could come at any moment, enters an unlikely character: a mysterious man who lost everything in his final battle five hundred years ago, and had awakened from a long, icy slumber. Unable to fight any longer, the man, Willem, becomes the father those kids never had, caring for and nurturing them even as he struggles to come to terms with his new life, in which he feels the pain of helplessly waiting for his loved ones to return home from battle that his 'Daughter' once felt for him so long ago. Together, Willem and the leprechauns gradually come to understand what family means and what is truly worth protecting in their lives.

....

Willem Kmetsch (ヴィレム・クメシュ Viremu Kumeshu)

Voiced by: Ryohei Arai (Japanese); Micah Solusod (English)

Willem is the main protagonist and only "alive" emnetwiht (人間族エムネトワイト, 'human'; from Anglo-Saxon) being in a world where humans have been driven to extinction by creations they call "Beasts". Known as the Black Agate Swordmaster, he is later found under a frozen lake after being cursed to petrifaction for five hundred years by the beast Eboncandle. He continues his fight against the Beasts, this time as the adviser to the "Fairies," since his body is too damaged to wield a Weapon after his battle with Eboncandle. As an active fighter he once wielded Percival, a mass produced version of the Dug Weapons (once called Carillons).

Anime opening

Found with a search query of anime "light novel " legendary swords talismans orphanage