We were passing along Fireplace Road in Springs, Long Island. Jackson Pollock had met his death on that road in 1956, just eight years before. I was travelling with Bryan Robertson, then the Director of London’s Whitechapel Art Gallery and author of a book on Pollock, the first major monograph on any of the artists of the New York School. Hans Namuth, the photographer who took the iconic images of Pollock at work, was at the wheel and we were heading north for the house where Lee Krasner, Pollock’s widow, still lived.

I had been there before, in my imagination, watching Hans’ film of Pollock working out there in the long grass in the sun and wind, seen from below dripping paint on glass against a blue sky, in the distance the clapboard house and barn.

Lee was not what she had been branded. In New York myth she was the monstrous Gorgon who guarded the Pollock estate. In person, though feisty, she was a warm and generous-spirited woman whose circumstances had forced her to stand up to the male prejudiced elite of critics and dealers hell-bent on devouring her husband’s life work. Then after Pollock’s death, she flourished into becoming the major artist who was always in her as a young woman before she became subsumed into being the protector of the alcoholic genius she married.

Sitting with drinks outside in that balmy August night, we listened to crickets and watched shooting stars. In that landscape I felt a closeness to the spirit of both her and Jackson’s work. In later years she became a good friend. I stayed with her in her New York studio in the Hotel Adams on 86th Street and there she talked of her first meeting with Jackson and the tragic ending of his life after he had failed to join her in their long-term dream to visit Europe. We discussed her work and tried to solve the problem created by her tacking canvases to the wall so that she could work on a hard surface. It meant trying to resist working to the very edge to avoid creating an impossible loss of image when the canvas was later fixed to stretcher bars.

Due to some rift between the artists, unexplained to me, Lee was not on speaking terms with the Motherwells and one had to tread a cautious path between them. I had met Robert Motherwell and his wife Helen Frankenthaler earlier when they visited my studio in London. Seeing them again in their home and studios in New York and Provincetown, I got a better sense of their creative relationship.

Helen’s masterpiece Mountain and Sea (1952) hung in their Manhattan brownstone. It was painted in oils that were extended with copious amounts of white spirit soaked into cotton canvas. It surprised me that Helen had remained loyal to oil paints as, by the time I was in New York, safe water-based acrylics were available (the new Magna paints had been unsafe, causing the tragic death of artist Morris Louis in 1962). Bob Motherwell stated his resistance to acrylics forcefully, claiming he was “of the wine culture not the Coca-Cola culture”. He was prone to somewhat stubborn announcements of that kind and once in my presence said, “All the greatest painters were fat”. I couldn’t help thinking he said that because he was concerned about his own increased girth.

In Provincetown I was struck by how much the sea had affected their painting. Bob was the youngest member of the New York School but 13 years older than Helen, who he saw as a kind of protégé, and so the interaction between the two artists’ work was unexpected. I had always seen his work as weighty and sombre – for instance, the whole ongoing Elegy to the Spanish Republic series and, one of my favourites, Je T’aime No. 3 with Loaf of Bread (1955). Whereas now, especially in his series Beside the Sea, I could see a youthful spirit in his work that was, I am sure, as much influenced by Helen as their home by the sea.

From the beginning of my life in New York I enjoyed the friendship of Barnett Newman. Of all the artists I got to know he was the one who was the most outward going and friendly. If he hadn’t heard from me for a while he would call to see how I was getting on and suggest a meeting. He and his wife Annalee were inseparable. She was a schoolteacher, I think maybe a headteacher by that time, and she had been his moral and financial supporter throughout his life. When he died she wore his hat for a long time afterwards.

A favourite meeting spot was a popular Irish restaurant called Billy’s on First Avenue by 53rd Street. With its red and white chequered tablecloths and hearty food it provided a convivial venue for dinnertime meetings. On at least two occasions after eating, Barney would insist on walking me down First Avenue to a spot where, looking towards the East River, a huge factory chimney rose up into the dark sky. I don’t remember if he told me what it was, and I doubt I could find it again if it still exists, and he didn’t explain why we had to visit it because with a mischievous gesture he explained everything. Even the outside world imitates Barnett Newman.

In spite of, or perhaps because of, his old school manner, he was huge fun to be with. He was a great raconteur, quite a joker and a self-parodist. He sported a monocle and, when his mood or the weather dictated, spats. He liked to confide in me over painting techniques. He was delighted that I shared with him a preference for wooden rather than metal stepladders in our studios, and he generously introduced me to Lenny Bocour, the pioneer of the manufacture of artists’ acrylic paint.

Barney and Annalee’s apartment on West End Avenue was surprisingly formal. The sitting room was set up rather like a business waiting room. He and Annalee had been through tough financial periods in their life and he wasn’t prepared to sell out easily; he took immense trouble to vet the seriousness and calibre of potential collectors of his work before he agreed to sell to them. He was working on sculpture during those years and the latest works, columns set in irregular bases cast in bronze, were on formal display. He told me about the problems attending the creation of his Broken Obelisk (1963–69). He wanted the smallest possible joint between the two main elements; the solution came from an engineer who advocated the use of an extreme type of steel only used for airplane wings.

One day soon after the publication of a book on the New York artists, lavishly illustrated with photographs, he was delighted to share the story about one particular shot of the interior of his studio, which showed his paint tubes and brushes regimented in neat order on his worktop. He said that he supposed the authors of the book were attempting to illustrate that his working practice echoed the apparent formality of his paintings. He said, in reality, that he worked in absolute chaos and every week Annalee came by to clear the mess up for him.

Staying with Lee in Springs afforded the opportunity to meet up with Mark Rothko, since he was spending the summer with his family in Amagansett. Here was Rothko in domestic circumstances with his wife, Mell (who, incidentally, always referred to her husband as Rothko), and their two children. We had lunch and went to the beach to bathe. He had a small studio in their rented beach-side accommodation but he wasn’t too proud of what he had been doing there, as I remember some smallish unfinished works. Even then, and this was before I got to know his New York studio, I had the impression he was ill at ease with the bright sunlight of the sky and beach.

I got to know Rothko better through visits to his studio over the following three years. It was then an old coach house on the Upper East Side. The vestibule had a table on which was a scale model of his current project, the future Rothko Chapel in Houston, configured as a hexagonal plan. Beyond, in the main studio, too big and dark to discern its own floor plan, were three walls especially built to represent the walls of the chapel. Huge mural-sized canvases covered each of the walls in Rothko’s typical deep reddish-black hues.

Anyone visiting Rothko’s studio in those days may remember, as I do, the gloominess of its ethos. He has been described as being clinically depressive but maybe that came later. I remember him rather as a melancholic man. His studio matched this mood, being lit from above by a lantern skylight masked by elaborate systems of Holland blinds with which he constantly fiddled to adjust the stream of daylight falling down from high in the centre of the space.

He explained how he believed that most people seem to make the wrong assumption that with a dark painting you have to give it more light, when in fact by putting more light you get nothing extra back. It was Rembrandtesque and indeed Rembrandt was on his mind for much of the time. As he reclined on a divan bed against one side of the studio he expounded on his admiration of the white luminous ruffs in Rembrandt’s portraits and how they registered as visual clouds of light hovering against their dark backgrounds. Mark seemed to feel that his own spiritual home was back then and he was living in the wrong place in history. That was how I remember him; three years after I left New York he committed suicide.

Abstract Expressionism is in the Main Galleries from 24 September until 2 January 2017.