Migration and Immigration. Wonderful features of the human species.
If you have Scandanavian ancestors, you could have some relations to the Vikings and the Norse who roamed the waters and controlled the Northern expanses of Europe long before British or even Roman rule.
Also the Gauls who predated the Romans were an exploring group of people and their heridtary influences can be seen in Germans, French, Belgians, and a good portion of Scandanavia and the Slovak Countries.
Remember as well that genetics does not recognize borders and countries as they are today. The European Baltic States were different in the 1800s as were they in the 1600s. Almost every century saw some sort of change in the human geography of those areas.
If you were able to find written history from your ancestors, you could shore up a lot of your concerns or questions..