Argthtjtr

Our Hero, James the Summoner, has the Counter-Summoner archetype and is 2nd level. This grants him the following abilities:

Counter-Summon (Su) ... a counter-summoner can attempt to counter a conjuration spell cast by another creature. He must identify the
spell being cast as normal for counterspelling. If he does so, he can
attempt to counter the spell as a swift or immediate action. To
counter the spell, the counter-summoner must attempt a dispel check as
if using dispel magic. If the spell being countered is a summon
monster or summon nature’s ally spell, the counter-summoner gains a +5
bonus on the dispel check.

Detect Summons (Su) At 2nd level, as a swift action, a counter-summoner can target a single creature he can see and determine
if it has been summoned to its current plane from another. This
ability reveals whether the target has been conjured by a conjuration
(summoning) or (calling) spell, and allows the counter-summoner to
attempt a Spellcraft check to identify the spell that conjured it
(using the same DC as if the counter-summoner had witnessed the spell
being cast).

His arch-rival, Alice the Druid, summoned d3 badgers to assault our Hero in advance of their inevitable combat.

Assuming that James successfully identifies the spell that was used to bring these rabid badgers into being, can he attempt to dispel them or is he restricted to doing so only if he witnesses the spell being cast in the first place?

No, it doesn't work as dispel magic

When the ability says "like dispel magic", it is referring to the check to see if the ability works or not from Dispel Magic:

To counter the spell, the counter-summoner must attempt a dispel check as if using dispel magic.

You make one dispel check (1d20 + your caster level) and compare that to the spell with highest caster level (DC = 11 + the spell’s caster level). If successful, that spell ends.

The Counter-Summon ability is clear that it works as the spell is being cast:

He must identify the spell being cast as normal for counterspelling. If he does so, he can attempt to counter the spell as a swift or immediate action.

The confusion on this ability always comes up because it mentions that it works with either a Swift or Immediate action. You can only use Swift actions during your turn, so how could you use this ability as a swift while the spell is being cast?

Well, it's simple, really. Most summoning spells take a full round to be cast, including summon monster and summon nature ally. This means that the caster begins casting the spell on his turn, and will only finish casting the spell at the start of his next turn, giving you time to force a concentration check by damaging him or even attempt to counterspell the summoning.

But, if the spell was already cast and the creature is summoned, the ability does not work as dispel magic against them.

Also note that the Spellcraft skill may only identify spells as they are being cast, it does not allow you to identify the effects of spells. Though, you may use Knowledge (Arcana) to attempt to identify the school of magic (necromancy, conjuration, transmutation, etc) that created a magical effect while using Detect Magic.

As for Pathfinder Society specific rules, I found nothing worth mentioning, other than a thread asking wether or not you can counter-summon against a summoning trap. You can't.