Most designers would never use RGB in place of Pantone because they serve two totally different purposes.

RGB is used for web design. It's got the best color range of any color model (though you'll notice that it still can't recreate any color, and it can't even recreate every Pantone color), but those colors are created through light, which mixes colors very differently than pigment. Anything that you print out (posters, business cards, etc.) will use pigments, not light. So RGB can't be used for printing purposes. If you've ever printed a file that was designed using RGB values, it was most likely run through an RGB to CYMK converter prior to printing.

So, in printing, we really have two options - CYMK or Pantone. When you print something using your printer at home, you are using a CYMK printing process. Let's say that you are printing this picture of a frog. In order to keep costs low and simplify the printing process, you generally aren't actually printing any green or orange ink. Instead, your printer is basically printing different parts of the same image four times, layering tiny cyan (C), yellow (Y), magenta (M), and black (K) dots across the page in a way that tricks your eye into thinking that those colors are mixed together (pointillism works off of the same principle).

Meanwhile, Pantone colors are pre-mixed. They are usually only used by designers because they have to be specially selected and ordered. Each Pantone color has a very specific formula that ensures that each batch of color is identical, kind of like how Home Depot mixes its paint colors. In contrast, CYMK can result in slight color variances depending on when and where it was printed.

For the average person, it probably doesn't matter, but for larger companies, they want consistency. A company like McDonald's might want to ensure that their French Fry containers are always the same shade of red. You've probably printed an image that came out looking funky because your printer was low on ink. Pantone doesn't have that problem because it only has to ensure the quality of one ink color, instead of trying to calibrate and balance the quality of four or more cartridges.

Lastly, Pantone can print certain kinds of ink that CYMK can't, like neon or metallic ink.

-----------------------------------------

ELI5 explanation: CYMK is like trying to drawing a picture using four colored pencils; Pantone is like ordering specific premade colored pencils to draw with.