Portsoy was created a burgh of barony in 1550. A burgh of barony is a type of Scottish town (burgh) in which the land title was given to a landowner who, as a tenant-in-chief, held his estates directly from the crown and allowed him to hold weekly markets. It was Mary, Queen of Scots who offered the burgh to Sir Walter Ogilvie of Boyne with all the usual privileges.
The first harbour was built at around the same time and replaced in 1692 by Sir Patrick Ogilvie, the 8th Laird of Boyne with one built entirely of stone. The Lairds of Boyne had been influential in the area since the late 1300s and built the clifftop Boyne Castle which was replaced by the Palace of Boyne in the 1570s.
The Palace was still being used as a grand house in the mid 1700s when the Ogilvies' support for the Jacobite cause led to their being stripped of their lands. Its ruins can now be seen two miles (3.5 kilometers) to the east of Portsoy.
The harbour built at Portsoy in 1692 was a vast breakwater on the seaward side and a number of quays. Large stones were set vertically as it was said this made them less likely to be washed away by the sea. The Old Harbour can still be seen today and is mostly the harbour that was built in 1692.
Portsoy’s first harbour was considered to be the safest in the North East, which meant that it had a thriving trade with both England and the Continent.
The new harbour was built between 1825-28 to meet the demands of the herring boom and the volume of trade going through Portsoy. Throughout the nineteenth century a herring boom brought further prosperity to Portsoy, with a herring fleet totalling 57 boats at its peak, though usually averaging 40-50 boats in harbour employing 108 resident fishermen and boys.
The harbour was washed away in an extremely violent storm in January 1839 and was only rebuilt in 1884 to hold 12 vessels of 100 tons, though most ships visiting the harbour were much smaller. The main imports were coal and bones from the Baltic, and the chief exports grain, herring, and potatoes.
Towards the end of the nineteenth century the Portsoy fishing fleet moved to the larger harbours of Macduff and Buckie and today Portsoy's harbours are primarily used by pleasure craft and creel boats catching lobster and crab.
Portsoy was particularly well known for its marble, cut from a vein of serpentine which runs across the braes (hills) to the west of the harbour. The dark green or reddish stone Portsoy Marble was greatly appreciated for its beauty and was used in the construction of parts of Louis XIV's Palace of Versailles.
In 1859 a railway line opened in Portsoy which became part of the Great North of Scotland railway system in 1867. Once the railway opened, trade increased significantly at Portsoy. The opening of the Moray Firth coast line in 1886 served trains that run between Aberdeen and Elgin.
The line was nationalised in 1948, but later recommended for closure by Dr Beeching's report "The Reshaping of British Railways". The railway closed on 6 May 1968. The station building is now used as a Scout hall and was renovated between 2011 and 2016.
Aside from industries around the harbour, other industries in the town at the end of the nineteenth century were a small ropework and a bone mill, and in the neighbourhood there was a wool mill, and the Glenglassaugh distillery which was built around 1875. The trade in Portsoy's early days was very varied, and included the import of coal for domestic fires and the export of locally produced thread and linen to England.
A particular speciality was locally quarried green Portsoy marble or serpentine. This was extracted from a quarry to the west of the town, and some of it can be found in Louis XIV's Palace of Versailles in France. Portsoy marble is still worked locally and a range of products are on view in one of the warehouses overlooking the harbour.