Modi also made an unscheduled trip to a village, Chango, and said he was “deeply touched by the impromptu reception and their joy”.
Dressed in olive green, the Prime Minister interacted with men from the Indo-Tibetan Border Police (ITBP), the Dogra Scouts and the army at Sumdoh.
Sumdoh, some 330 km from Shimla, is located on the border of Kinnaur and Lahaul-Spiti districts.
Mingling freely, Modi went up to the soldiers holding a plate of sweets. At least one soldier offered him a sweet in return, much to the joy of everyone.
He was accompanied by National Security Adviser Ajit Doval and the Army chief, General Dalbir Singh Suhag.
The Prime Minister also met personnel of the General Reserve Engineering Force (GREF), a wing of the Border Roads Organisation (BRO) that maintains highways, at Sumdoh, an official said.
Modi then spent some time with the locals in Chango village in Kinnaur district, known for its delicious apples.
Villagers taken aback by the Prime Minister’s arrival raised slogans hailing him. Dressed in warm clothes, Modi spent time with the residents of the village including women and children.
He also posed with everyone in a group photograph, with the mountains providing a majestic background.
The Lahaul-Spiti district, spread over 13,835 sq km, is a place of remote, untouched beauty with just 31,528 people. The climatic conditions are harsh as much of the land falls under a cold desert where the mercury drops below minus 20 degrees Celsius during winter.
The Prime Minister had celebrated his first Diwali after coming in power in 2014 with soldiers posted in Siachen. In 2015, he was at the India-Pakistan border in Punjab.