Heat the oil in a 12-inch skillet over medium heat. Reduce to medium low and add the shallots and 1/2 tsp. salt. Cook slowly, stirring occasionally, until soft and golden, 15 to 20 minutes. Put the potatoes in an 8-quart pot and add enough water to cover by about 2 inches. Add 1/4 cup of salt. Cover the pot and bring to a boil over high heat. Reduce to medium low and simmer, uncovered, until the potatoes are tender, about 20 minutes. Drain, return to the pot, and stir gently over low heat to dry them. Dump the potatoes into a bowl and then pass through a ricer or food mill back into the pot. Whisk in the melted butter, creme fraiche, and milk, adding more milk as needed to reach your desired consistency. Set aside 3 Tbs. of the shallots and stir the rest into the potatoes. Season to taste with salt and pepper. Transfer to a serving bowl, top with the reserved shallots, and serve.