Hi,

thanks for your inquiry. I hope I can provide you with some useful answers.

I'm confused about the concept of a programmer for microcontrollers. [...] I understand this to mean that the SPI pins behave as programming pins using a certain programming protocol when certain conditions that put the microcontroller into programming mode are met. So, the programmer is nothing more than hardware that knows how to speak the programming protocol.

First of all, J-Link is far more than just programmer, it is also an advanced debug probe.
J-Link is used for debugging of embedded target devices, you can download your application to the target and debug it in a supported IDE (everything listed here and additionally every IDE which makes use of a GDB Client) like you would do it with a PC application and a PC IDE (for example MS Visual Studio or Apple XCode). For example, it is possible to set breakpoints, to read / write Memory and Registers and to halt and continue the application at any moment.
We also provide the J-Link software & documentation pack, which contains lot of sophisticated tools and technologies for embedded development, which can be used free of charge with J-Link.

If so, why can't an Arduino, or some other general purpose hardware, also speak the programming protocol for ARM?

There are standard interface "protocols" to communicate with a target MCU (e.g. the ATSAMD21G18 on the Arduino Zero) like JTAG and SWD. SPI is a protocol which is used to communicate with an external SPI flash without interfering with the target MCU.
A J-Link can be used to communicate with every device which is based on a supported processor core listed here. Additionally, J-Link explicitly supports the devices listed here (>7500), which means that J-Link handles all necessary special behavior of the listed devices and that J-Link can program their internal Flash.

Also, what is so special about programmers that it has to be hardware and cannot just be software as long as the microcontroller pins can be hooked up properly and the software can control those pins?

In theory, it is possible to debug a device solely with a PC. Some simple debug probes only provide the convenience that you do not have to connect each pin to your PC. Instead, you connect the pins to the probe and control the Pins via USB.
As an advanced debug probe, J-Link provides 4 major advantages to that approach:
1) It is much faster (e.g .Write / Read speed to RAM / FLASH)
2) You can directly start working on the actual project (developing an application for the target) instead of implementing a lot of debugging logic on you own.
3) You can use our Development Tools
4) You can debug your target application from within the supported IDE of your choice

If this really is the case, why are Segger products so expensive? What is their secret sauce that seems to give them some sort of monopoly?

"Secret sauce:" A significant amount of knowledge and man-years which were used to develop the hardware, the firmware, and the corresponding PC software of the debug probe.
Therefore, we think that our products are priced appropriately(The prices are far away from the maximum you can spend on a debug probe), and we also offer a J-Link EDU-Version (approximately 60€) for hobbyists and education purposes.

The Nordic nRF51822 device series is listed as a supported device: segger.com/jlink_supported_devices.html?m=Nordic%20Semi#sel

If you want to know more about our development tools, I would suggest to check out our website, for example:

Systemview (Runtime analyses of the behavior of an application)
Ozone (Standalone debugger)
RTT (Technology for real time communication between target and host PC)
J-Scope (Realtime visualization of static and global variables located in the RAM of the target device)


If their are any specific questions, do not hesitate to ask me.

Best regards,
Niklas


Literature:
JTAG
SWD
SPI
J-Link