"I said 'see you later, Rusty,' and that was over 70 years ago," Middleton said.
The day he saw Johnson again came July 4, when Rusty, 79, traveled from his home in Arlington, Wash. to Juneau after looking for his brother for 50 years.
Rusty remembers spending days as a young boy visiting Middleton's farm.
Photo by Amanda Gragert Joe Middleton, left, escorts his younger brother Rusty Johnson out of Juneau International Airport on Wednesday, July 4. The brothers were reunited in Juneau after more than 70 years.
With an age gap of about 15 years, the brothers were close but distance pulled them apart. Middleton moved from their home state of Minnesota but wrote to their mother once a month.
"I always kept in touch with Mom. I would write her letters to let her know where I was and what I was doing," Middleton said. "Well, she moved a couple of times, and I lost track of her."
Middleton moved to Alaska in 1946. He said he had always wanted to come to Alaska because he likes "wild places and open spaces." He's worked as a bush pilot, airplane mechanic, trapper, commercial fisherman and logger.
In losing touch with his mother, Middleton lost contact with his family as well. He is the oldest of eight children, three of which are alive - Middleton, Johnson and their youngest sister Margaret.
Johnson said he was about 10-years-old when Middleton moved but he never forgot about his oldest brother. He moved to Washington in 1950 and worked as a long haul trucker, driving from coast to coast. Johnson's wife of 55 years, Irene Johnson, said she started the search for her husband's family about 50 years ago.
"It was a round about way we found Joe. His mother had a brother she lost contact with, and I worked on finding him for a while," Irene Johnson said.
"I worked with a bunch of little family clues and matched birthdates and places of birth and it all started coming together. I thought I found Joe, and we thought he'd be about 95. Rusty thought he was surely gone but I didn't want to give up."
In April, Irene Johnson sent Middleton a letter explaining who she was and listing names of people he would know if he were indeed Rusty Johnson's brother. Middleton showed the letter to his friend Donna Sheridan.
"He gets bills and magazines but it shocked me to see something addressed to him that was handwritten," Sheridan said. "I asked him what it was and he said 'it got me quite upset.' I looked at the letter and asked him if he knew those names and he said he knew every one of them."
After Middleton's wife and daughter died years ago, he thought he had no direct family members still living.
"I gave up on ever knowing anyone from my family," Middleton said. "I had to call him up."
Not being able to hear well, Johnson nearly hung up the phone on Middleton before his wife intervened.
"He was about to hang up, and I came running out of the bedroom and said 'don't hang up, it's him,'" Irene Johnson said.
"I was so shook up that I handed the phone to my wife," Rusty Johnson said. "I couldn't believe it was really Joe."
The two men started to talk on the phone regularly and soon decided to meet. Once the Johnsons booked plane tickets, Sheridan offered them a place to stay at her home in Mendenhall Valley and to take them to Middleton's apartment in Downtown Juneau each day of their trip.
"Irene and I were pretty committed that these two boys needed to see each other," Sheridan said.
Irene Johnson said that the experience seemed surreal when it was in the planning stages. After her husband experienced a few recent medical emergencies, she was worried the meeting wouldn't take place.
"We didn't know if it was going to work but I think it was meant to be," Irene Johnson said. "We have four girls and this has been good for them to see their dad get his wish. They've heard about Joe all their lives."
Shortly after arriving to Juneau, the two men settled in for a visit, talking about people they both knew and memories they shared.
"It's a good thing we've got some time. I've got a book of things I want to ask you," Middleton told his younger brother. "I'm just a kid you know. I thought I'd work on my second childhood but my memory doesn't work like that anymore."
Johnson shared a story of his oldest brother driving a car into a fence gate before being interrupted.
"I tell you what. You're right but you've got the wrong place," Middleton said with a smile.
Sheridan said that the two men have been visiting as much as they can, stopping to take a few naps to regain their strength before again revisiting their memories.
"It's more like we're twins rather than 15 years apart," Rusty Johnson said. "When he feels bad I feel bad and vice versa. We may not have seen each other in 70 years but it's evident we're brothers."
The stories continued about life on the farm and the strong resemblance Middleton had to a brother Johnson had known well.
"This is the most I've seen him talk in a long time," Irene Johnson said. "This is such a happy, loving time for both of them in their advanced years. And now, even though the visit will be over in one short week, they both have many more phone calls to look forward to. It's been a rewarding walk."