Self-determination is the principle of international law that a people should be able to choose their political status for themselves. In practice, it has generally been extremely vague and under-recognized. The main problem is that self-determination conflicts with the ideas of territorial integrity and sovereignty. If a small region of a country wants to secede, but the national government does not recognize it, what then? It seems that most people, whether elites or the general public, see this kind of thing as a tough, thorny issue where distant observers don't really have a basis for taking a side. This means that the status quo of strict territorial integrity is widely enforced in practice, and attempts at self-determination rarely get international recognition.
I haven't looked at serious practical arguments on this (they actually seem hard to find) but my sense is that stronger norms of self-determination would lead to better governance. First, we need to recognize that historical borders have no intrinsic moral importance. In some cases they were drawn through processes of conquest or elite dealmaking that was totally illegitimate from the point of view of the broad population, and even when the historical drawing of a border was legitimate, that may not mean anything for how that border should be drawn today. The characteristics of particular national borders have no innate moral significance, just like the status of a particular family as the ruling monarchy has no innate moral significance. All that matters is whether the political arrangements benefit the local population and the international system going forward.
Second, there is a clear basic logic that government works better when people can make decisions about it. Democracy is better than autocracy because people can choose their leaders. By very similar logic, self-determination should be better than territorial integrity because people can choose their polities. This matters in a direct sense, as people are more likely to make decisions which favor their interests, and it also matters in the indirect sense, where governments will be incentivized to do better if they have to deal with a risk of secession. Presumably there is no systematic empirical evidence on how governance improves when people can create or join a new country, but on priors, I don't see why it would be significantly less than the degree to which governance improves when people can pick new political leaders.
Attempts at self-determination can be violent and illegal. But the same thing has frequently been true for attempts to achieve democracy. Many people think that democratic movements such as those of 1776 and 1848 were justified even though they entailed bloodshed and violated the decrees of the existing regimes. We can apply the same kind of logic to efforts at self-determination.
One can identify cases where efforts at self-determination are not well grounded in the interests of the people. Sometimes people have wrong opinions about what kind of governance would be best for them. But again the same thing is true for democracy. Sometimes people pick the wrong policies, but if we went around supporting autocrats in every case where we thought they could pass better policies, the consequences would ultimately be bad. When people vote for the wrong candidate, we say that they're wrong but the results should be respected on principle; the same can apply when people vote to join the wrong country.
Self-determination may lead to a fragmentation of the world into smaller and smaller polities. This may seem detrimental to global immigration and trade, but in reality it would probably come alongside an increase in regional frameworks similar to the EU which solve such international issues at a broader scale. So we could simultaneously fragment and coalesce in different respects. This will certainly change the character of the international system, but wouldn't be obviously better or worse. Fragmentation would allow rich areas to better insulate themselves from demands to redistribute their wealth to poorer areas, and this is somewhat troubling, but in practice few polities seem willing to split along wealth lines.
Regardless, self-determination in many cases only amounts to a desire to switch allegiance from one existing country to another. In these cases, the number of polities just doesn't change.
The main problem with self-determination so far is that associating every ethnic group with a right to have their own polity is inherently destabilizing wherever ethnic groups are closely mixed. Abkhazia is a notorious example - the Abkhaz people were only a minority of the population of Abkhazia, but in pursuit of their goal of having an independent Abkhaz nation, they committed ethnic cleansing and expelled huge numbers of ethnic Georgians. Now the majority of people in Abkhazia are ethnically Abkhaz. While the principle of self-determination does suggest that Abkhazia as it is today should be recognized as independent, applying it in such a case would create a bad moral hazard (unless we make them pay significant compensation, prosecute perpetrators and allow refugees to return). Clearly, we cannot grant the right to self-determination to every ethnic group, because so many of them are minorities even in their own lands. So self-determination should be recognized along territorial and political lines, not ethnic ones (although majority ethnic composition can still be a good reason for residents of some particular territory to pursue self-determination).
There is room for some sensible disagreement on these issues, but the existing political regimes will predictably fall on the side of under-recognizing attempts at self-determination. Any country which sets a liberal precedent for another country opens a legal and rhetorical vulnerability to losing some of its own territory to self-determination movements. Governments of existing countries are incentivized to over-emphasize territorial integrity at the expense of humanity. To make things better, we should consistently emphasize self-determination.
Now there is certainly plenty of difficulty in actually figuring out how to implement the principle of self-determination in a legally and politically sound manner. I'll leave that to others, and I don't claim that doing this kind of work is a good EA cause area. I just believe that, as citizens with a minor role in setting global norms and policies, our default reaction to efforts at self-determination - such as those in Kurdistan, Palestine, Crimea, Catalonia, West Papua, Hong Kong, and elsewhere - should be to acknowledge that they have a basic pro tanto right to their aspirations, and only secondarily consider legal and practical arguments against them.