A "favicon" will generally appear near a browser's address bar when a user is visiting your website.
However, these icons can appear in a number of places beyond the address bar, including alongside the website's name on a browser tab and on the "home screens" of mobile devices. As a result, the favicon should provide a simple and straightforward visual cue to help a user quickly identify your website amongst other websites.
At the very least, the Web Standards Group recommends creating a favicon.ico file and placing it in the root directory of your website (this particular icon does not need to be specified with a meta tag in the HTML).
This favicon.ico file should contain two images of different sizes: 16x16 and 32x32.
This provides a bare level of support for browsers and devices.
To create a better user experience across a number of platforms, including varying pixel densities, devices, and operating systems, refer to the open-source favicon cheat sheet.