When Future Dabura turns the Z Sword into stone Future Trunks drops it, resulting in it breaking, and since this happens while it is stone, this results in Future Old Kai's death.
Video game stories
Future Old Kai appears when Gohan goes to the Sacred World of the Kais with Future Supreme Kai and Future Kibito to free the Z Sword. Gohan can not pull the sword out due to not being the 'chosen one', so he and Future Supreme Kai go to search the Other World for Future Gohan.
After a search that led to the two meeting Future Pikkon, Future Bardock, and Future Goku, they return and Future Gohan easily pulls out the Z Sword, in contrast to Gohan's own experience. Just as the Z Sword comes free, Gohan completely smashes it, much to the dismay of Future Supreme Kai and Future Kibito. Gohan tells them that the sword itself does not matter, and when Future Supreme Kai replies in argument, the voice of the Future Old Kai brings their attention to him. Future Old Kai then asks Gohan how he knew, while Future Goku and the others wondered who the old man is. Future Old Kai offers to unlock Future Gohan's potential. Future Gohan agrees, saying he will do whatever it takes, which the ancient Kai thinks is good in a young person.
After Future Old Kai had unleashed Future Gohan's hidden potential, he asks them how this will work since Future Gohan is still dead, remarking that the others did not put thought into it. Then, in the same manner that his counterpart had done for Goku, Future Old Kai gives his life to Future Gohan. Also, Future Pikkon and Future Bardock are given one day in the living world for them to fight against Future Super Buu on Earth. Before they leave, Future Old Kai also gives them Potara earrings, saying they can fuse with this.