Clicker Training, what is it? [Archive]

Bay Mare

24th Nov 2007, 06:15 AM

Clicker training is positive reinforcement. You ask for something, they try it, you click and reward.

It's an excellent way of training a horse from on the ground, long reining right through to advanced dressage movements under saddle. Certainly from doing it with Saff we've gone from a horse who will work in the school if she had to to a horse who practically drags you into the school and is extremely happy and willing to learn.

Although you start off clicking and treating when you're first training you eventually move onto a schedule of reinforcement so that not every try gets a click (otherwise you'd have a really chunky horse!). You can have a fixed schedule whereby they get a treat every 5 'tries' for example but a better schedule is a variable schedule where they don't know when they're going to get a reward. This means that you may treat after 3 tries, then 7, then 4, then 10 for example. You can sort of equate this to a human in an amusement arcarde playing on the machines. If you get a small win you'll keep on trying and because you don't know when the next win will be you won't lose enthusiasm (unless you don't get a win at all of course!).

You do have to make sure that your timing is right for clicker so that you're clicking for the try and not for something 2 minutes later ;) I've found it easier to learn a tongue 'cluck' rather than using a clicker as you don't have to fumble with the clicker and it's easier to use if you're riding, doing in hand etc.

Have a look at Alexandra Kurland's site (which probably explains it better than me!):

http://www.theclickercenter.com/

AengusOg

24th Nov 2007, 07:24 AM

Here's a good idea..............why don't you throw your clicker away and use your voice to reward every time the horse gets it right.

The clicker trainers I've met have been the most amateur, clueless individuals I've ever met in the horse industry; clicking away all the time through vet/farrier visits, conversations, and all the time their horse is trying to eat fingers or clothing, ever on the lookout for the elusive treat which they may or not get, depending on which click sequence is being applied by their ruler this time.

It's just another money-making scheme for the cranks who promote it.

Use your voice; it's the most rewarding sound a horse can hear.

KAB

24th Nov 2007, 08:43 AM

Sorry I cant offer any advice - but I do know Novicenic has done a lot of clicker training with her boy and it appears to have worked wonders. I am sure she will offer you advice when she logs on. :)

Bay Mare

24th Nov 2007, 09:07 AM

The clicker trainers I've met have been the most amateur, clueless individuals I've ever met in the horse industry; clicking away all the time through vet/farrier visits, conversations, and all the time their horse is trying to eat fingers or clothing, ever on the lookout for the elusive treat which they may or not get, depending on which click sequence is being applied by their ruler this time.

It's just another money-making scheme for the cranks who promote it.

How rude! And talking of clueless ....

Maybe you've just seen it being done wrongly and badly like I did when I first came across it. Seeing it being done properly is nothing like you describe. Maybe you should do some research on learning theory and then see what you think.

Correct clicker training/positive reinforcement actually helps with muggy horses and, done correctly doesn't need to be used every hour of every day. The very reason that you do a variable schedule of reinforcement is so that they can't predict the reward.

Maybe you should tell Alexandra Kurland, Becky Holden, Heather Moffett and the rest of the good trainers who do clicker that they're cranks :rolleyes:

Est

24th Nov 2007, 09:34 AM

Here's a good idea..............why don't you throw your clicker away and use your voice to reward every time the horse gets it right.
Use your voice; it's the most rewarding sound a horse can hear.

I don't appreciate being called a crank and would rather move on to a more constructive discussion.

I agree with you that voice reward is great, and I also build in a lot of Tanya Larrigan's suggestions to my training.

The difference between voice and clicker (whether with a clicker or a tongue click or a whistle or whatever is being used) is that clicker is very quick and precise. I can mark the exact behaviour that I want. Sometimes the behaviour takes a second or so, no more. I wouldn't always have time to say "Good boy!" or "Good girl!" before the behaviour was ended and something else was happening. The clicker means that I can let them know something was good instantly.

It also makes me think about my behaviour as a trainer when I'm doing clicker. I have to be very focused to ensure I notice even the tiniest effort to get something right, and it improves my timing in giving praise for that. Of course, anyone training a horse should try to do this - I have just personally found that clicker training has challenged me and improved me in this because it is so precise.

It's worth adding that clicker can be teamed with *any* reward (voice, treat, wither scratch etc), and that clickers cost around £1-£2 (mine was free from Eukanuba with some dog food!!). I have never bought any books on clicker, and do all my work from free articles on the internet, so it doesn't have to cost very much at all.

Pilgrim5, a good site basic introduction is: http://www.equusite.com/articles/training/trainingClicker02.html#introducing

Also, for more detailed information this site has lots of free articles to read: http://www.clickersolutions.com/articles/index.htm

You can also see some awesome videos on YouTube. This thread has the links:
http://www.newrider.com/forum/showthread.php?t=108726

Hope that is some help in your query :)

wonkeywoody

24th Nov 2007, 12:31 PM

It also makes me think about my behaviour as a trainer when I'm doing clicker

...you should have always thought about your behaviour, you shouldnt need the clicker to remind you!!!

I find a 'good boy' and a lovely rub gets a better result for reward because it encourages relaxation and thinking about what they have done instead of being focused on food.
I can see it would work for some, but thats why there are so many different training methods. Each to their own.

Bay Mare

24th Nov 2007, 01:30 PM

But that's where you're missing the point. Once they're established in clicker they DON'T focus on food. The treats are used at the beginning of training to reinforce the 'click'.

It also makes me think about my behaviour as a trainer when I'm doing clicker

...you should have always thought about your behaviour, you shouldnt need the clicker to remind you!!!

I think that you misunderstand. You can always be better as a trainer and clicker can help your timing and your focus. It's easy to miss the small 'tries' when you're working with horses and especially when you have a goal in mind. Clicker focuses your mind on the small things, the small things that can make a big difference.

Each to his own at the end of the day. I didn't like clicker when I first saw it being done (badly). I thought that it was 'dumb' and pointless. After seeing it done correctly, though, I was converted. The click is very specific as long as you get the timing right which can be very helpful in training. I still give 'good girl' and a scratch, the two can be used together. It's about being open minded, making sure that you have the facts and using what works for you and your horse.

Becky Holden trains horses up to advanced dressage moves (piaffe, passage etc) with the help of clicker. She's certainly not an amateur and she's certainly not clueless!

AengusOg

24th Nov 2007, 03:00 PM

Hi Bay Mare

What I said may have been rude, but it's also true.

I actually do lots of research on learning theory, it's part of my job, and I wouldn't feel comfortable giving an opinion otherwise.

I would welcome an opportunity to discuss their training techniques with Kurland, Holden, and Moffet, as I believe they could easily achieve the same results without a clicker.

AengusOg

24th Nov 2007, 03:12 PM

Here's the thing Est.........I don't recall describing you as a crank.

Clicker training was originally marketed to the dog training industry (hence your find in a dog food pack), then some bright spark, with a weather eye for a quick buck, broke into the horse industry.

It's a superfluous, money making gimmick. I've never met anyone who has taken it seriously, or persisted with it beyond a few weeks; maybe I should get out more.

Est

24th Nov 2007, 05:23 PM

@ Aengus Og - I promote clicker to people where I think it would be beneficial - you said people who promote it are cranks. Ergo, I'm a crank :D But I'm a happy crank... I use clicker amongst a lot of other methods including, as mentioned, voice training. *Of course* it's possible to train using other methods (or even multiple methods) - there's more than one road leads to Rome, as they say.

Interestingly, I also have a background in learning theory and I agree that a purely behaviouristic approach would have its limitations. I believe it's important to consider the *process* of learning too. I think in some ways, we're arguing for the same thing - just from different angles!

@ Wonkeywoodey - I completely agree that we should always think about what we're doing when training. Maybe my original post didn't express that well enough - I was trying to say that! My point was simply that clicker really shaped up my timing, because the feedback is *so* quick as to whether my timing is right or not.

Also interestingly I have an example from tonight where clicker was useful. My mare needed her leg bandaging and, having been on box rest for over a week, was acting a bit disgruntled! She decided to hold her leg up in the air. I initially praised verbally for lowering it, but often she was already moving her leg back up as the words came out (i.e. I was praising the wrong thing). So I grabbed my clicker and clicked for any movement downwards. Within 10 seconds her hoof was flat on the floor and she stood perfectly for the bandage. She had a nice wither scratch (her personal favourite reward!) afterwards. I know there are lots of other ways of achieving the same thing, and clicker was just the one that worked for me tonight.

I think you've summed it up well: "each to their own", and I would encourage anyone interested to find out more and decide for themselves if it's something they could use or not.

neen

24th Nov 2007, 05:33 PM

Who on earth makes money from clicker training? You'd have to sell an awful lot of £1 clickers. You might end up making a buck, I suppose, but I doubt it would be a quick one.

I've used it to train a shy pony to let me touch his head. It worked brilliantly. I only used it for 2 days, because after that there was no need; we'd achieved what we'd set out to do.

I didn't use a clicker -- I just made a clicking sound with my tongue. Can't get much cheaper than that!

india

24th Nov 2007, 05:41 PM

Clicker training is positive reinforcement.
I have nothing against clicker training (except under certain circumstances) but I think it's very misleading to suggest to the OP that it's a 100% positive reinforcement training method.

I can't understand why some clicker training fans appear to have a problem with admitting negative actions are also used. Personally, I don't think a method exists that relies purely on positive reinforcement to train a horse.

Zingy

24th Nov 2007, 06:29 PM

Each to their own.

I completely agree :D

I have horse who didn't get his head round clicker training at all. He didn't understand a treat being used as a reward as what he really wanted was praise from me.

One of the ponies enjoyed it to a point, but figured the treats weren't worth putting a great deal of effort in for :rolleyes: So you could never get to the stage of not reinforcing every time as he gave up.

My other pony loves it. It makes him really think about what I'm asking him to do and keeps his focus far better than verbal praise or anything else.He is very food orientated, but doesn't mug me for treats as he used to do when I got him and he had been rewarded for good behaviour using food. Without the clicker, that reward as far as he was concerned was quite random and therefore if you had food it had to be his. I could no doubt use other methods, but when he enjoys it and it makes him think and we achieve something from it, then why not use it?

So yes, it's very definitely whatever works for your horse. One size doesn't fit all, never has and never will.

Est

24th Nov 2007, 06:47 PM

Zingy

24th Nov 2007, 07:21 PM

[QUOTE=Est;1476339@ Zingy - you could always use your voice/praise as the reward - the click is just a very precise marker of a desired behaviour, which says "something good is coming!" My mare responds much better to wither scratches and fuss, while my gelding prefers Mr Johnson's Equiglo Herbal Treats! :D[/QUOTE]

He didn't get the click/ treat connection though :rolleyes: The sessions went something like:

Click.............oh, a treat. Nice. Thank you

Click.............oh, a treat. Nice. Thank you

Click.............oh, a treat. Nice. Thank you

I gave up after a few sessions :o I figured if he couldn't understand the click/ treat connection, I would never stand any chance with the behaviour/ click connections :D

He certainly worked better with other training methods!

india

24th Nov 2007, 07:53 PM

@ India - I think BM means that clicker (as in the act of clicking) is positive reinforcement. It's adding something which makes a particular behaviour more likely to happen again.

But I agree 100% with you about training programmes as a whole - I struggle with the idea that it is possible (or even desirable) to *only* use positive reinforcement. It's great to be able to shape a behaviour through positive reinforcement and it would always be my preference, but when I lunge, I ask my horse disengage his hind quarters through my body language - negative reinforcement! And yes, if a horse barges me it gets sent out of my space PDQ - positive punishment (adding a reinforcer which makes a particular behaviour less likely to re-occur).

BM - would you agree? Do you use any other type of reinforcement? Any idea what Becky's take on this is?

I agree Est, that part of clicker training is the positive reinforcement area, but there are also parts of training method that include negative areas..for example the differential reinforcement stage. As the OP is asking what clicker training is, I still think it was misleading to suggest it's a 100% positive reinforcement method, especially to someone who hasn't any experience of clicker training and what it involves.

This thread included my views and also an example of how I interpreted differential reinforcement, it would be great to read views on it. :)

http://www.newrider.com/forum/showthread.php?t=124362

As I've already mentioned, I haven't got a problem with clicker..infact I think it can be a great addition to any training method – but there are both negative and positive areas included and I believe it's only fair to point this out to OP.

:)

Est

24th Nov 2007, 08:47 PM

I think you've made a really interesting point :)
I'm having to think my way through this, so bear with me if I ramble!

Billy doesn't know the criteria for getting a bronze or a silver star (or at least he doesn't appear to). I don't know (and therefore I guess he doesn't know) how many factors have to change for a silver star (use more colour? apply perspective? do more realistic landscapes?) and whether he has to achieve all of them at once, or just one at a time, or simply start to make an effort to do some of these things. Also, his paintings are consistently at bronze level with some silver, the praise is undifferentiated and always there... and then the praise suddenyl vanishes - it is by chance that he does a silver star painting later on and happens to get praise again.

I think that this kind of situation could certainly occur with training a horse and just like Billy the horse would be confused as to the criteria needed for success. If there is no expectancy of success, then motivation (energy directed towards a purpose) will be lacking. The training will grind to a halt! The trainer will have been using negative punishment (removing a reinforcer making a particular behaviour less likely to re-occur) probably without even realising, and, like capalldubh said, the desired behaviours will reduce in frequency/intensity as a result, making it clear that's what's happened.

This would be an example of poor clicker training (I believe). The key differences are that a clicker trainer should:

1. Train for *one* thing - perhaps a longer time, or a greater effort, or a bigger distance - never multiple factors at once. Billy should know that he is working on using more colours in his paintings. (The teacher knows that once he is consistently using 10 colours in his paintings, they will start to work on his use of perspective.)

2. Train in small increments, not a "jump" to the next grade. So the teacher wants Billy to use 10 colours in his painting to get a silver star and he currently uses two, which is bronze star standard. He uses a third colour and gets immediate praise. A-ha! he thinks, and uses a fourth colour... Except a good clicker trainer would be praising him for looking over at the paint pots, then for starting to stretch out his hand for another paint pot, then for touching the paint pot... A very clear shaping of behaviour in small, easily understood steps with the smallest "try" rewarded, not just waiting until he quite randomly happens to use all 10 colours and then saying, Bingo!

3. Only move to an intermittent reinforcement schedule when the desired behaviour is fully established - i.e. Billy is always producing bronze standard paintings with two colours, and now gets occasional comments. He knows that he and the teacher are working towards silver stars now by gradually using more colours, and that's where the regular praise is to be found. No confusion, just plenty more opportunities to make a "try" and get it right. If one of his "tries" is wrong (he paints on some PVA glue instead of paint - maybe adding *anything* to the painting is good), this behaviour can be ignored (extinguished) and if the teacher has handled the situation right, he'll be motivated (and confident enough) to try something different.

So yes, I agree that it is possible for intermittent reward to become negative punishment - but I would add a proviso that it is the case only if it is being done badly, as in Billy's example.

Ok, ramble over :p Your thoughts? Have I made any sense?!

JOJOBA

24th Nov 2007, 09:08 PM

I really like clicker training. My reasoning, as a purely pleasure rider/owner, is thus;

1) I make mistakes. Sometimes Im unclear. However, the clicker makes it harder for me to be unclear. My horse learned a lot quicker with a clicker than just with me - the reason being, the clicker is consistant.

2) Its FUN! My horse enjoyed it, and so did I. He learned that doing a certain thing got him a click-treat. So he went from being grumpy introverted horse, to interested using-his-brain horse.

3) It's time spent with your horse. Whether you are hugging him, chatting to him, clicking him, whatever, it's all just low pressure fun time hanging out. And the horse enjoying your company.

4) I talk a lot. When I ride I often chat to the horse. I recently rode a mare I didnt know, and I was chatting away to her, and asked her to do a school movement. She tried a few times and didnt quite get it. Then she tried and for a split second got it right, and without thinking I clicked her. Her ears went forward and she relaxed, and got it right every time from then on (without the clicks). I rang her owner later and said 'do you do clicker training with your horse?' and she said she had started recently and how did I know? :cool:

5) I know people who use it as a calming / refocusing tool. If the horse has a fright, loses concentration etc they just do a target excercise to get the attention back.

I think the argument around positive reinforcement is a valid one, and I would say that my limited clicker training is based around positive reinforcement. Personally I ignore incorrect behaviour and reward good behaviour. However, there is another behaviour - BAD behaviour. That one I negatively 'reprimand'. ie, Im not going to allow a horse to nip me.

To use an example;

My horse used to mug me for sweets.
If he mugged me I ignored him.
If he bit me he got a smack / chased away (I dont tolerate biting).
If he turned away and left me alone he got a click-treat. And eventually just a click.

My horse no longer mugs me for treats (or rather, no longer mugs his new owner. I hope!). If he thinks I have a treat he will turn his head away and wait quietly for it. Or for nothing, on his unlucky days ;).

I just thought Id give the perspective of an amateur who has found it useful in their day to day handling of horses.
Like all things, it needs to be coupled with some horse sense and common sense. Clicker training doesnt have to be a total training method on it's own. Ask someone like Pink's Lady, who has been doing it for ages :p. I love the mix'n'match approach to horse handling - use whatever works best for you and your horse.

As for it being a money making gimmick - my clicker cost me £1.25 from our local pet shop (rip off, they're usually cheaper!). That was all. I never read any books on it, I never did any courses in it. In fact, I did it without any reading up online. I have a dog and some common sense, I just bought a clicker and gave it a go! Worked pretty well for me :o. Most people who try it arent intending to take over the world, some people just want to have a horse who is a bit easier to handle, or one who understands them a little better, or some other small goal.

xxx

redcomet

24th Nov 2007, 10:01 PM

its a brill little invention. it took me about 10 mins using a clicker to get holly to lift her leg for a treat. now i just point at her hoof and shell lift the leg. the only downside is she will lift 1 foot then the other whenever she fancies something or hears a rustling in your pocket! :D

india

25th Nov 2007, 01:18 AM

Thanks for your views Est, they make perfect sense and I can really see where your coming from with regard to training one thing at a time, in small increments and rewarding the slightest try...without a doubt, all positive areas of clicker training. :)

Here's my thoughts on 'Billy's' other points though......

Billy had reached his bronze standard and was working his way towards a silver star by being taught to progressively incorporate other colours into his painting. He wouldn't have been confused by this because they'd been introduced gradually and he'd been receiving continuous praise through out for his efforts.

Because of the extra colours he'd learned to use, he realised they where working towards a silver star, which was fine....that is, until the teacher began to ignore some of his bronze standard efforts that she'd previously praised...........although Billy was getting regular praise for using the extra colours, it wouldn't have helped him to understand why the praise was being with held from some of his bronze 'tries' and this would have confused him...albeit to a lesser degree.

As differential reinforcement works by rewarding only the best 'tries' and ignoring or with holding treats from the rest...that's one area of clicker training that can't fail to include some degree of negative punishment.

Would you agree? :)

stormy's mum

25th Nov 2007, 02:50 AM

It's a superfluous, money making gimmick.(quote)
i dont think so i train service dogs and all of them are trained using a clicker it works well and its not expensive they listen very well and are happy to work . doing very specific tasks

Waikato Valuta

25th Nov 2007, 06:05 AM

I love clicker training. and one of the best things about it is it's free!!!

You can use anything that makes a sound, you don't have to buy an expensive one. And sure you can buy book and attend clinics like anything else but there is also enough information on the interest for free to get you started.

When done well it works wonders. but when done badly it can be VERY bad. If you don't make sure you insist on 100% cooperation, e.g. let the horse mug you for food, it wont work. Infact it can turn a normal nice horse into a little monster.

Est

25th Nov 2007, 10:27 AM

WV - agreed! And I can see the same thing applies to *any* training method done badly (the horse may not mug for treats [assuming that's what the clicker trainer was using as the reward], but it may barge, or actively resist the trainer, and so on. I think that many training methods, if done well, are a useful "tool" in the trainer's "skills toolbox", and if done badly will be unrewarding and potentially damaging to the horse. So I think you are absolutely right that any trainer should try ensure they are at least basically competent in the method chosen at that time, and try to improve themselves continuously to a good level of skill. A good point :)

India - yes I can see where you're coming from with this. (Poor Billy! :p ) I would like to put forward a slightly different slant on differential reinforcement before trying to capture some more rambling thoughts! (I'm really enjoying this discussion BTW - it's very thought-provoking - thank you :) )

Differential reinforcement is "as it is" - it is only +P or -P if the horse/Billy decreases the behaviour in question as a result of the DR. I feel that you're right: Billy's example does indeed show a teacher who is using negative punishment (although probably without meaning to), because in this case they are removing a reinforcer (praise) and it is making Billy confused and upset - he will make less and less effort at his paintings (decrease his bronze star behaviour) because his motivation to do that behaviour is trickling away - soon he'll be playing truant and mugging people outside the local shopping centre!!

But if DR is done correctly (and I do agree that not everyone manages this!) it shouldn't lead to a reduction in frequency/intensity of the behaviour, so it wouldn't be -P (nor should it affect motivation/confidence - difficult to measure, but I think a change in motivation or confidence would have a visible effect on the behaviour we see).

The other difficulty with Billy is that he's a tough example to use for clicker, because the secondary reinforcer or bridging signal (bronze star) is given for an entire completed painting some time after he's finished painting it, which of course shouldn't happen with clicker unless the trainer's timing is extremely bad! Capturing a fully-formed behaviour is the weakest clicker method (or even teaching method) because the learner won't understand the component parts that went into the achievement and will be less able to recreate the behaviour (or if they do manage to recreate it, it will most likely be through chance and not from true learning).

My concluding ramble is, "Would DR be a "bad" thing anyway in learning?" i.e. when done correctly as agreed (!) but regardless of what name we put to it (DR, -P, etc).

Would another human analogy be ok? At junior school I needed praise for doing joined up handwriting. It took me a while to establish this, and so I was praised every time I did it. But by senior school, I had established joined up handwriting and was working on good use of headings, sub headings etc in assignments. If I had *still* been getting praise 100% of the time for joined up handwriting, I would have felt a bit like, "Hey, we've covered this ground before", and wouldn't have felt at all motivated to improve anything else such as my use of punctuation. I would also start to feel a bit bored and unchallenged. Joined up handwriting was my new norm and the learning had moved on quite naturally.

So the flip side is that if DR is done correctly, introduced only for established behaviours and always introduced gradually, then it shouldn't cause any negative feelings or confusion and therefore shouldn't decrease the behaviour or the motivation... so shouldn't (in good cases like this) be -P.

But I also agree that if introduced abruptly, or before learning is established, or for multiple criteria etc, then yes it would be very likely to result in confusion, upset etc and decrease the behaviour, and therefore be -P. And I wonder (going back to Aengus Og's post) if this is where a lot of people come unstuck and end up giving up?

I suppose to summarise.. cos a post this long needs a summary :D ... it all depends on the trainer. I think that how they manage DR decides the effect on the horse, which in turn decides whether it is -P or not.

Congratulations if you've read this far!! :D

Daffy Dilly

25th Nov 2007, 12:22 PM

On a slight tangent, went into Pets at Home earlier to buy another clicker, and got asked "how old is your dog?" "he's almost 16, but it's not a dog, he's a horse". I got some funny looks :p

stormy's mum

25th Nov 2007, 05:12 PM

On a slight tangent, went into Pets at Home earlier to buy another clicker, and got asked "how old is your dog?" "he's almost 16, but it's not a dog, he's a horse". I got some funny looks :p

lol i go to the pet store and buy vet wrap and i get asked what did your horse do and i say its for a dog

Francis Burton

25th Nov 2007, 06:40 PM

I have nothing against clicker training (except under certain circumstances) but I think it's very misleading to suggest to the OP that it's a 100% positive reinforcement training method.

I can't understand why some clicker training fans appear to have a problem with admitting negative actions are also used. Personally, I don't think a method exists that relies purely on positive reinforcement to train a horse.
I think it comes down to the fact that if it's not 100% positively reinforcing then you are doing something wrong. It can happen inadvertently, of course, but it doesn't have to - it's not a required or inherent part of the technique.

As mentioned before, it very much depends on the horse.

A horse that is used to being told what to do all the time, never allowed to have an opinion and has a tendency to "shut down" might be affected really badly by having even a single treat withheld (or delayed slightly), and it would end up having a -P punishing effect. On the other hand, a well-adjusted horse and one that had been doing CT for some time might respond to a withheld treat in the opposite way. It might make that horse try harder and end up strengthening the behaviour. In this case, what you call "differential reinforcement" (I would probably call it "variable schedule reinforcement") will have a purely positive effect - and there'd be no negative repercussions as far as the horse was concerned. However, if you modified your shaping plan too quickly by switching from a click-to-treat ratio of 1:1 to 15:1 say, it is likely that even the more motivated/CT-savvy horse will give up long before the 15th click. Then it will have a punishing (or extinguishing) effect, and the trainer would have failed to use CT properly.

Careful, observant trainers who found they were dealing with the former type of horse would simply not use variable schedules - i.e. they would click and treat every single time.

Hope this clarifies the issue.

Francis Burton

25th Nov 2007, 06:48 PM

Here's my thoughts on 'Billy's' other points though......
Are you saying that every horse is like young artist Billy?

As I described in my previous post, the reaction to a "missing" reward would depend on the horse and how motivated it was feeling. One would hope that Billy's teachers would recognize the potential negative impact of ignoring his effort and try hard to avoid that. A good CT trainer would do the same with a horse and only use variable schedules if it didn't have a punishing effect.

Francis Burton

25th Nov 2007, 07:17 PM

He didn't get the click/ treat connection though :rolleyes: The sessions went something like:

Click.............oh, a treat. Nice. Thank you
Click.............oh, a treat. Nice. Thank you
Click.............oh, a treat. Nice. Thank you

I gave up after a few sessions :o I figured if he couldn't understand the click/ treat connection, I would never stand any chance with the behaviour/ click connections :D
It sounds like you were trying to teach what the click means by associating it with a treat that was given a short time after the click. Is that right? If so, that would be a typical "classical conditioning" scenario (like Pavlov's dog associating the dinner bell with being fed). You can teach the meaning of "click" this way, but in my experience it is more effective to do it with a different kind of conditioning ("operant") in which the horse is rewarded as the result of a specific (and fixed) action that it performs itself. By simply clicking and treating randomly, the horse isn't sure what it did to get the reward. Instead, you can set up a situation where the horse almost can't fail to perform an action and therefore elicit the click+treat.

The one I like to use is the horse touching a cone with its nose. You plonk a cone on the floor right in front of the horse and (assuming the horse isn't scared of it!) he will almost certainly be curious and put his head forward/down to sniff it. That's when you click and treat. The first couple of times the horse may be appear surprised by the sound, but he should also be pleased by being given a tidbit. (If not, he is either not hungry or not particularly food-motivated - some horses are like that.) After a few successful goes, the horse typically shows another expression - as if a lightbulb has lit up - an "Aha!" moment when he realizes that something he has done has caused the food to appear. This is the start of the horse's discovering that he can control events and make rewards appear out of the air - except that it's you that's really controlling events, of course. ;)

I'm not sure why it is the case that by linking the click+treat to an action the horse somehow makes the association between click and treat more quickly - but it does seem to work!

JOJOBA

25th Nov 2007, 07:24 PM

Francis Burton

25th Nov 2007, 07:53 PM

If a horse is not food motivated and instead is looking for the approval of his owner, is there any reason why you shouldnt click-scratch? Then after a while you just click and he associates that with 'mum is pleased with me'?
I dont see clicker training as being food based at all - just pleasure based. Whatever gives the horse pleasure and good feelings is what the reward should be?
Absolutely, Jojoba - whatever works!

Also, the clicker part of CT training just makes the timing of the "Yes!" message more precise, allowing you to reward specific actions more easily. There is nothing to stop you using a voice signal instead - as long as the horse understands that it means "Yes!" and has pleasurable associations.

Pink's lady

25th Nov 2007, 10:13 PM

Like jojoba said, I use clicker training but not as a sole means of training.

Like just about every method of training (apart from The School of Common Sense) clicker training don't work particularly well on it own, and if you do try to, it becomes rather 'cult-ish' and obsessive.

Done wrong it doesn't work but if it IS done badly it's probably better than a large number of other methods done badly.

AengusOg - you may have looked at the theory, presumably badly, but clearly never done or seend any of the correctly done practical part:rolleyes: That's like seeing one idiot run their horse into the ground trying to join up and dismiss all NH as rubbish and pointless - a rather narrow minded point of veiw.

Clicker training was originally marketed to the dog training industry (hence your find in a dog food pack), then some bright spark, with a weather eye for a quick buck, broke into the horse industry

esp as clicker training was initially used to train dolphins, not dogs. Horses are as good, if not better, than dogs at clicker training when taught properly.

india

25th Nov 2007, 11:22 PM

Would another human analogy be ok? At junior school I needed praise for doing joined up handwriting. It took me a while to establish this, and so I was praised every time I did it. But by senior school, I had established joined up handwriting and was working on good use of headings, sub headings etc in assignments. If I had *still* been getting praise 100% of the time for joined up handwriting, I would have felt a bit like, "Hey, we've covered this ground before", and wouldn't have felt at all motivated to improve anything else such as my use of punctuation. I would also start to feel a bit bored and unchallenged. Joined up handwriting was my new norm and the learning had moved on quite naturally.
Just read through your post Est...wow, very thought provoking!!!!:)

Your human analogy was excellent and I must admit the learning programme certainly progressed more naturally in that than it did in 'Billy's' example. For that part, I agree that if differential reinforcement is done correctly it shouldn't cause the horse any undue negative feelings.

Having said that, the fact remains that negative punishment is included in clicker training and for that reason I still believe it's misleading for anyone to suggest it's an 100% positive reinforcement training method.

Sorry this is brief, but it's been a long day and I'm very tired. :o.........

india

25th Nov 2007, 11:58 PM

Francis Burton

26th Nov 2007, 07:46 AM

I disagree, as clicker training includes negative punishment...I can't see how it could possibly be classed as an 100% positive reinforcement method.

It would be interesting to read your views on this information Francis:
http://www.clickandtreat.com/FF01.htm
Well, by that token a horse who is pottering about by himself is being continually negatively punished because he is not being constantly rewarded. I (and horses) can live with that. :)

What I meant, but probably expressed rather poorly, was that CT needn't involve the use of aversives - anything that the horse finds unpleasant. Withholding a treat could have that effect, but it depends on the horse, the situation and whether CT is being done well or not. During training (in fact, all the time you're with the horse), not rewarding a horse for behaviours you don't want will cause them to decrease. Whether you call this "extinction" or "negative punishment" is semantic - it's a relative effect. The important thing is how the horse feels about what is happening to him. Is his mood positive or negative; is he eager or "sulky"? When CT is done right, the horse is in a positive emotional state - he doesn't find the training unpleasant. (That's not to imply that other forms of training are necessarily unpleasant for the horse, by the way.) It's a matter of balance. Does the horse feel punished? Is he thinking "I'm not enjoying this!" at any point?

To repeat: when CT is done right, the horse won't feel like Billy.

I think paragraph 1) in the article you cited is misleading. It appears to suggest that aversives are an intrinsic part of CT. In fact, they say aversives may be necessary at the "end of the learning cycle" as "corrections" - but rarely when "you use this sequence, correctly". It's not clear what they mean by sequence here: is it the sequence of clicks and treats, or is it the sequence of CT followed by correction? If it's the first (which does make sense logically), they seem to be saying that if C+T is done well, corrections are rarely needed. That's fair enough. However, the implication is that CT includes more than just clicking and treating - that it is a complete training philosophy rather than a specific method. They have defined it to include other methods, i.e. application of "corrections"/aversives. What we're discussing in this thread is the specific CT method, not the other methods which may or may not be employed as well.

Having said that, I don't believe that using CT alone is practical for training horses, although some people have managed to do amazing things using CT alone. I personally use negative reinforcement a lot! Hopefully, most is fairly subtle. Frankly, I can't imagine training without -R (but maybe that just means I lack imagination). However, when I do use CT to tackle specific problems I tend not to combine it with -R and pressure. For horses that are used to being told what to do, it can be a distraction and have a overall negative influence, so it's usually better to keep the CT session "pure".

india

26th Nov 2007, 11:28 AM

Well, by that token a horse who is pottering about by himself is being continually negatively punished because he is not being constantly rewarded. I (and horses) can live with that. :)
:confused:Could any one make a true comparison with a horse that's pottering about by himself to one that's under going a clicker training session (or any other training for that matter)..personally, I doubt it.

Although I'm obviously not as experienced as you are with the training method, I understand the initial stages of clicker training encourages rewarding the horse for every try. It's only later that negative punishment starts to appear and even then, it's not continuous.

To repeat: when CT is done right, the horse won't feel like Billy.
And likewise...I've already agreed with that in a previous post. ;)

...However, the implication is that CT includes more than just clicking and treating - that it is a complete training philosophy rather than a specific method. They have defined it to include other methods, i.e. application of "corrections"/aversives. What we're discussing in this thread is the specific CT method, not the other methods which may or may not be employed as well.
Maybe I'm wrong, but through reading different articles on Clicker training I got the distinct impression it was both a training philosophy and a training technology which definitely includes more than just clicking and treating to reach the desired result.

As the OP asked "clicker training...what is it" do you not think it appropriate to include both the positive and negative areas?....maybe not everyone does include negative punishment and aversives in their training....but the fact remains they are part of the overall clicker training method.

Out of genuine interest...can anyone provide a link that confirms Clicker Training consists only of positive reinforcement.

:)

Yann

26th Nov 2007, 11:45 AM

Out of genuine interest...can anyone provide a link that confirms Clicker Training consists only of positive reinforcement

I stand to be corrected but I don't think you'll find one. Most clicker training with horses involves some kind of physical suggestion to begin with which amounts to negative reinforcement, even if it later becomes a cue once the horse has cottoned on. Otherwise you're with the dolphin in the pool scenario, waiting and watching for the behaviour you want to occur by itself so you can reinforce it. I can imagine that would a) take rather a long time and b) be rather frustrating for the horse.

As to whether switching from a continuous schedule of reward to a variable one amounts to punishment or an aversive I have no idea. As I understand it you only do this once the horse begins to understand and look for the click as a good thing in its own right, if so I don't think this would be the case from its point of view.

From my dabblings I've found mixing + and - to be the most effective way to teach new things :)

Est

26th Nov 2007, 12:33 PM

Ok - I'm skiving from work and back into the fray!

Yann - you're right, some people do use -R (suggestion, pressure/release, etc) to induce the first stages of a desired behaviour (I've seen it happen as well), but that's not clicker training itself (which is click + reward)! And of course it's equally possible to shape the behaviour in small steps starting with something freely offered by the horse. I've never found a situation where there wasn't *something* I could use as a starting point, whether curiousity or something else, with no -R.

Waiting for a complete behaviour to happen ("capturing") is one method of training clicker and it remains true to the +R principle, but as you say, it's time consuming and probably quite boring too! As I've said in a previous post, in my opinion it's not good learning either. So we're in agreement there! :)

I feel very strongly that clicker training (i.e. click + reward) shouldn't be confused with the approaches and philosophies that other people start to build round it. Of course people can build entire training programmes around it, and yes these may incorporate elements -R, -P and even +P, but clicker itself (click + reward) is +R.

FWIW I also use different methods which include -R, and again as I've said in another post, I think that an effective trainer should have a variety of skills/methods/tools in their toolkit! So I do agree with you in most of what you've said :)

India - great link and I agree with most of it - sounds like a very common sense guy writing it.

The two bits I disagree with are:

By scientific definition, the removal of a desired reward is a "negative punishment."

I define -P as "the removal of a reinforcer which makes a particular behaviour less likely to occur". Failing to give a desired reward is only -P if it makes the behaviour less likely to occur - and clicker (click or click + reward) is trying to shape a behaviour to increase its frequency or intentisty - so quite the opposite! So a trainer could inadvertantly do -P, perhaps by moving to DR poorly, but DR itself shouldn't be -P unless the trainer is messing up somewhere!

Francis Burton, how would you define -P? Do you have a different take on it?

The second bit I disagree with is:

Needed corrections are almost always applied at the end of the learning cycle, after the dog has a firm knowledge of the behavior. When you use this sequence, correctly, you may be surprised at how rarely your dog needs corrections -- but there will always come a time when you need to tell your dog that obedience is not "optional."

Absolutely! But that's not clicker training (click + reward)! That's part of an overall training strategy which may or may not include elements of clicker training. That particular bit isn't clicker training.

Right, got to get back to work but I shall be back on tonight to read the next installment!! :D

Mistertron

26th Nov 2007, 05:18 PM

I can vouch that Est is certainly not a crank of any description! ;) but i'm only saying that because she bought me a Mars bar yesterday! :p

Anyhoo....I have seen how effective clicker training is watching Est work with Merrick. It's definately something i'd try if i get my own horse in the future.

It's all well and good that people have different opinions about this topic but some of the posts arent very constructive. The person that said 'each to their own' got it right'.:)

Francis Burton

2nd Dec 2007, 04:49 PM

Sorry for taking so long to get back to answer your question, India - it has been very busy here with teaching, student projects etc.

:confused:Could any one make a true comparison with a horse that's pottering about by himself to one that's under going a clicker training session (or any other training for that matter)..personally, I doubt it.
All I meant was, whether a horse is pottering about alone, interacting with other horses, or doing stuff with people, he/she is continually encountering opportunities to learn. The principles of learning are exactly the same in all these situations. That's the great thing about 'learning theory' - it helps you to understand why horses' behaviour is changed by their experiences, and there isn't a completely different set of rules for when they are "being trained". It's not the whole story, of course - recognizing horses' instincts and innate drives, having a sense of how they perceive us and the world around them, knowing their physical and mental strengths and limitations - these all help too.

Although I'm obviously not as experienced as you are with the training method, I understand the initial stages of clicker training encourages rewarding the horse for every try. It's only later that negative punishment starts to appear and even then, it's not continuous.
With respect, no it doesn't - as explained above. At least, it's only punishment if the behaviour you're trying to teach/encourage is actually discouraged!

Maybe I'm wrong, but through reading different articles on Clicker training I got the distinct impression it was both a training philosophy and a training technology which definitely includes more than just clicking and treating to reach the desired result.
That's fair enough, India. There's no denying that there is a range of different angles on clicker training "out there". It seems that one can't even rely on all articles to use exactly the same definitions! From what I have seen, articles specifically written about clicker training horses tend to be the more helpful ones for horse owners.

As the OP asked "clicker training...what is it" do you not think it appropriate to include both the positive and negative areas?....maybe not everyone does include negative punishment and aversives in their training....but the fact remains they are part of the overall clicker training method.
Well I guess that depends on how broad one defines the method of 'clicker training' to be! Do the 'Join Up' or 'advance and retreat' methods include punishment/correction? I would say not (negative reinforcement, certainly, but not punishment, if done correctly). However, I'm quite sure that folks who use these techniques also correct/punish unwanted behaviours as the need arises. That doesn't mean that punishment is an integral part of 'advance and retreat' or 'Join Up' any more than it is of 'clicker training'. I personally consider clicker training to be just another technique - "another tool in the toolbox" as others have aptly put it. There are indeed some people who have attempted to make 'clicker training' their main (even sole) modus operandi. I'm not sure how typical they are though - everyone I know who uses it, does so in a selective, targetted way.

Out of genuine interest...can anyone provide a link that confirms Clicker Training consists only of positive reinforcement.
Would having a link that says this would actually prove the point one way or another?

Anyway, to get a flavour of what's available, I did a Google search on "clicker training horse" and found some quite informative articles...

http://www.companyofhorses.com/pages/clickerbasic.html

"Training horses using ‘positive reinforcement’ is slowly becoming accepted. The problem is the word ‘positive’ gets misrepresented and attached to methods and trainers just because they do not advocate punishment. Beware, there is a difference!"

"It has to be said that there is no room for punishment, correction, anger, or impatience when using Clicker Training. It is a way of training that means you need to to ignore all the behaviours your horse will offer that you do not want. This includes incorrect or even bad behaviour. Unless your horse is totally unrestrained (free shaping) you will also be using negative reinforcement in one form or another. However,you must never mix clicker training with punishment as it can cause neurosis, a complaint far too many horses already suffer from."

(Here I think the authors are talking about positive punishment, and are making a clear distinction between it and ignoring/not rewarding unwanted behaviours.)

http://www.kbrhorse.net/tra/clickr01.html

http://www.theclickercenter.com/2004/guide/index.php

http://www.angelfire.com/az/clickryder/intro.html

"What is a Clicker Trainer?

This issue has come up before. A couple of months ago Bob and Marian Bailey and I got together on the phone and hammered out a definition we could live with, which I then passed by some of the autism folks and they approved it too. I have submitted it to William Verplanck in hopes that he will add it to the Cambridge Center for Behavioral Studies website glossary. here it is (I may have a slightly different wording at the office, this is from memory):

Clicker training is that part of operant conditioning that utilizes positive reinforcement and a conditioned reinforcer used as a marker signal.

In other words, our definition specifically excludes punishment. When you are punishing, you are not clicker training. You are punishing. You may be negatively reinforcing operant behavior but the use of the aversive under this definition says that that procedure is not clicker training.

So, punishers may be clicker users without being clicker trainers in this definition. Or they might be clicker training some of the time and conventional training some of the time...which will have an inhibitory effect on the overall rate of offered behavior in the subject, as well, of course, as the risk of fallout of various other unpredictable sorts, as we all know."

Hope this helps the OP and advances the discussion in a constructive and interesting way!

Francis Burton

2nd Dec 2007, 04:53 PM

I define -P as "the removal of a reinforcer which makes a particular behaviour less likely to occur". Failing to give a desired reward is only -P if it makes the behaviour less likely to occur - and clicker (click or click + reward) is trying to shape a behaviour to increase its frequency or intentisty - so quite the opposite! So a trainer could inadvertantly do -P, perhaps by moving to DR poorly, but DR itself shouldn't be -P unless the trainer is messing up somewhere!

Francis Burton, how would you define -P? Do you have a different take on it?
I don't think I could do any better than you have, Est.

The only point which might be added is that a distinction can be made between removing a reinforcer in such a way that the horse knows it is being removed (or frustrating the anticipation of a reinforcer), and the simple absence of reinforcer. The latter leads to something called 'extinction'. To my mind it is a fairly subtle difference, as the end result of both negative punishment and extinction is the lessening of a behaviour.

Some people do make a point of separating these ideas. For example, Melissa Alexander writes in:http://www.clickersolutions.com/articles/2006/horsetraining.htm

"Second, the quadrant description [R+, R-, P+, P-] doesn't include a fifth principle of operant conditioning, one that clicker training makes particular use of. This is the principle of extinction. In extinction, a behavior is weakened through the absence of any kind of reinforcement. For example, if no one answers your knock at a door, you will eventually stop knocking. If a horse can't get a treat by "mugging" your treat pouch, he will stop trying. Pairing extinction -- "ignoring" responses you don't want -- with R+ -- actively reinforcing the responses you do -- creates a powerful method for teaching a horse to do what you want."

Whether or not one considers extinction to be a separate principle/process, I agree with what you say about failing to give rewards in clicker training being -P only if it makes the behaviour less rather than more likely to occur.

india

3rd Dec 2007, 12:04 AM

Francis Burton

3rd Dec 2007, 11:54 AM

Francis, please don't think I'm trying to discredit CT because I'm not...I truly believe, in the right circumstances, it can be a valuable tool in anyone's toolbox...but from what I've read, my view still remains that no over all training programme could ever possibly be 100% positive.:)
About that we are 100% in agreement! :) Our difference in opinion appears to stem from whether CT is considered to be an overall training programme or a specific tool (like 'advance and retreat' or 'Join Up'). I view (and use) CT as a tool. It is entirely feasible, when using this tool, to use only positive reinforcement and extinction. If withholding a treat results in the behaviour you are trying to encourage being discouraged then you have allowed it to have the effect of negative punishment - but that needn't be the case. If moving to a variable schedule (what you call differential reinforcement) results in punishment, then you have gone too far too fast. Variable schedule reinforcement isn't a mandatory part of CT, but when it is used care should be taken to ensure it has a rewarding rather than a punishing effect.

I think the 'positivity' of CT can be judged in two ways. First, it can be measured purely in terms of outcomes. For a behaviour to be reinforced in CT (in contrast to methods that use negative reinforcement), it must be rewarded. In other words, if we see behaviours reinforced, we can assume that they were indeed rewarded. Although this is a useful benchmark, it represents a rather mechanistic viewpoint.

A second way to assess positivity is to see the effect of the training method on the horse's emotional state or mood. Is it raised or lowered? In my experience, horses positively enjoy CT. It produces a visible 'lifting of spirit'. Horses often appear more 'sprightly'. They 'perk up'. One sees increased levels of enthusiasm, engagement with the training, reduced levels of fear, and a willingness to offer new behaviours that can push the boundaries of what the horse considers 'off-limits' (it's great for "breaking the ice"). In some cases where you are trying to overcome a reluctance of the horse to do something, or even a phobia, the general mood lift and lessening of fear that occurs when you start to use CT can have a positive effect on a problem you are tackling even before you start targetting the problem behaviour directly.

I have never encountered a horse in a CT sesson that looked like he wasn't enjoying it at any stage (though I have no doubt that very poor application of CT could cause frustration and grumpiness). Of course we can never be sure exactly what a horse is feeling moment to moment, but I think we can have a pretty good idea of overall mood.

Take these couple of examples:
http://www.clickersolutions.com/articles/2002b/factors.htm

"To train a behavior, I would say the vast majority are trained with mostly positive reinforcement and extinction. I use a small amount of negative punishment, even when teaching new behaviors. I don't generally introduce it early in the learning phase -- though I might with some behaviors. (Loose leash walking jumps to mind.) I prefer to use positive reinforcement and extinction, but it's a never-say-never situation".
Can you think of examples where negative punishment might be desirable in a clicker training session with a horse? Frankly, I am struggling to imagine cases where it would be useful or necessary.

And:
http://www.clickersolutions.com/articles/2001/primer.htm

"A more accurate depiction of the relationship between the principles of operant conditioning and clicker training begins with the image of a pie. In clicker training, positive reinforcement is the largest piece, taking up perhaps two-thirds of the pie. The second largest piece is extinction. The third largest is negative punishment. Positive punishment and negative reinforcement are just two tiny slivers. The most important thing to note is that a complete, reliable training program can be composed entirely of positive reinforcement, extinction, and, to a far lesser extent, negative punishment".
Considering clicker training as an overall training programme, as this is stated to be, the proportions of the different 'principles' or components seem very reasonable to me.

Maybe you could advise again here.....

In the extract below (copied from your previous post), it states there's no room for punishment in clicker training...yet goes on to add that all unwanted behaviour should be ignored

"It has to be said that there is no room for punishment, correction, anger, or impatience when using Clicker Training. It is a way of training that means you need to to ignore all the behaviours your horse will offer that you do not want. This includes incorrect or even bad behaviour."

However, in this article written by Karen Pryor, who I belive is one of the founders of clicker training, she suggests that ignoring unwanted behaviour is a punishment in itself.....
http://www.clickertraining.com/node/101

"Twice, the dog offers a sit, and she clicks and treats. The third time, the dog sits, but the trainer waits a bit longer, and the dog jumps up on her. She folds her arms and turns her back. As she does that, the dog cringes backward to the floor, as if it had been struck. Was that "punishment"? To the trainer, no; she just briefly removed her attention and what's so bad about that? To the cowering dog, yes: that really hurt. Punishers, like reinforcers, are defined by the receiver, not the giver. I am sure the trainer thought she was just communicating, not punishing"
I agree that ignoring unwanted behaviour has a lessening (i.e. punishing) effect on that behaviour. What I'm less sure about is the extent we can equate a dog's emotional responses to removal of attention to those of a horse. I don't know dogs that well, but I do know they view us rather differently than horses. If we turn our back on a horse, does it cower? Why does the dog cower and the horse not? Is it because dogs consider us to be part of their pack, so there is much more of a social aspect to our interactions? Perhaps doggy people would care to comment!

Although Karen Pryors article relates to dogs, wouldn't the same principles also apply when clicker training horses - if so, why are these quotes so contradictory?
Yes, I do think the same general principles of 'learning theory' can be applied to any mammal. However, I also think there are differences in the biological relevance of certain actions to each species and it is very much a case of punishers and reinforcers being defined by the receiver. If my ignoring a behaviour or temporarily removing attention resulted in the horse showing the emotional equivalent of "cringing backward to the floor", then I would be surprised and worried!