When the Muslims fled persecution at the hands of Quraysh, and came to Madinah for safety. Not long after settling, the Muslims made pact(s) with various tribes in Madinah. One of the first who joined this pact was the Banu Nadir tribe. Part of the agreement was that those who agreed to abide by the treaty, that they would not help anyone against the Muslims. Be that physically, financially or any other way. Another important point written in the pact was that if an enemy were to attack the Muslims they would all come together and protect the Muslims or vice versa.
The Muslims and Jews in Madinah had good relations. There was mutual respect from both sides. They had a lot in common in their religions given that Islam as a religion acknowledged all the Prophets from the Torah. The friendly relations continued until the Quraysh started waging war against the Muslims.
In doing so, the Quraysh also brainwashed some of the Jews, to spread hate and propaganda, so to disunite the various communities in Madinah. In this, they succeeded. In a short while, from various tribes living in harmony, it went to hatred and killing.
With the enemy Quraysh, not succeeding in killing all the Muslims, or revert them back to their former faith, idolatery. They now sowed discord. Some of them started slowly siding with the Quraysh against the Muslims. When the battle of Uhud took place between the Muslims against the Quraysh, we learn that the Banu Nadir leaders gave over information to the Quraysh on the weak spots of the Muslims in the battle.
With this information, the enemy would have been able to know where the Muslims were, where to attack, and where to hide, lay in wait to kill the Muslims. The Banu Nadir also incited Arab tribes to attack the Muslims.
Al-Bayhaqi’s Sunan al-Kubra:
Abu Abdullah al-Hafiz – Ismail bin Muhammad al-Sha’raani on the authority of his grandfather – Ibrahim bin al-Mundhir al-Hizami – Muhammad bin Fulayh – Musa bin ‘Uqbah on the authority of Ibn Shihab said:
This is the hadeeth of the Messenger of Allah (peace be upon him) when he set out to the Banu Nadir to seek their help to settle the blood money of the Kilaabiyeen [i.e. Banu Kilab] and they had spoken secretively to Quraysh when they went to Uhud to fight the Messenger of Allah – peace be upon him – and incited them and informed them of their [i.e. the Muslims’s] weak spots…. [the story continues to tell the story of how the Prophet sat resting on the wall and how they conspired to kill him] (Al-Bayhaqi’s Sunan al-Kubra, Hadeeth no. 17,207.)
“The cause of the war [against Banu Quraidah] was due to their breach of the treaty. As for al-Nadir [i.e. Banu Nadir], the reason [for their banishment] is due the following reason, which was mentioned by Musa bin ‘Uqba in “al-Maghazi”. He said: al-Nadir spoke secretively to Quraysh and incited them to fight the Messenger of Allah – peace be upon him – and informed them of their weak spots [in the Muslims defense] …” (Ibn Hajar, Fath, volume 7, page 421)
Moulavi Cheragh Ali:
“Ibn Ockba, an earliest biographer of Mohammad, died 140, says, – the cause of the expedition against the BANI NAZEER was this: that they had INSTIGATED THE KOREISH TO FIGHT AGAINST MOHAMMAD, and had reconnoitred the weak points of Medina.
Ibn Mardaveih, and Abd-bin-Hameed, and Abdu Razzaq have related traditions to the effect that, ‘after the event of Badr, the Koreish had written to the Jews of Medina to make war upon Mohammad, and the Bani Nazeer has resolved to break the compact.
(Vide Zoorkanee, part II (vol. 2), page -96-97) (A Critical Exposition Of The Popular “Jihad” – [Calcutta: Thacker, Spink And Co., 1885] by Moulavi Cheragh Ali, page 109 – 110 (footnote 3))
Besides this treachery committed against the Muslims, they also further on one occasion tried to assassinate Prophet Muhammed (p). When the Prophet (p) went over to ask them to pay the bloodwit (blood money) of which they owed, they agreed to pay it. But at the same time commanded one of their people to go on top of the roof’s house and throw a huge stone on the Prophet’s head.
THE DEPORTATION OF THE B. AL-NADIR, A.H. 4
According to what Yzaid b. Ruman told me the apostle went to B. al-Nadir to ask for their help in paying the bloodwit for the two men of B. Amir whom Amr b. Umayya al-Damri had killed after he had given them a promise of security. There was a mutual alliance between B. al-Nadir and B. Amir.
When the apostle came to the about the bloodwit they said that of course they would contribute in the way he wished; but they took counsel with one another apart, saying,
‘You will never get such a chance again. Who will go to the top of the house and drop a rock on him (T. so as to kill him) and rid us of him?’
The apostle was sitting by the wall of one of their houses at the time. ‘Amr b. Jihash B. Ka’b volunteered to do this and went up to throw down a rock. As the apostle was with a number of his companions among who were Abu Bakr, Umar, and Ali news came to him from heaven about what these people intended, so he got up (T. and said to his companions, ‘Don’t go away until I come to you’) and he went back to Medina.
When his companions had waited long for the prophet, they got up to search for him and met a man coming from Medina and asked him about him. He said that he had seen him entering Medina, and they went off, and when they found him he told them of the treachery which the Jews meditated against him. The apostle ordered them to prepare for war and to march against them (670), Then he went off with the men until he came upon them (680). (Ibn Ishaq, page 437) 
Kitab Al-tabaqat Al-Kabir – Ibn Sa’d:
“They said: O Abu al-Qasim, we will do as you like. Then some of them went into secret consultation and decided to act treacherously against him (Prophet). Amr Ibn Jihash Ibn Ka’b Ibn Basil al-Nadari said:
I shall ascend on the roof of the house from where I shall throw a stone down (on him). Thereupon Sallam Ibn Mishkam said: Do not do it. By Allah, he will come to know what you intend to do with him. It would be a violation of our agreement with him. The Apostle of Allah, received information of what they intentended to do.” (Kitab al-tabaqat al-Kabir vol. 2. Pg 69) 
Kitab Futuh al-Buldan:
“The Prophet once accompanied by Abu Bakr, Umar and Usaid ibn Hudair came to the Banu an-Nadir who were Jews and solicited their aid for raising the bloodwit of two men of the Banu-Kilab ibn Rabi’ah who had made peace with him and who were killed by Amr ibn Umaiyah ad-Damri. The Jews intended to drop a stone on him but the Prophet left them and sent them word ordering them to evacuate his city [Yathrib] because of their perfidy and violation of covenant. The Jews refused to comply, and announced hostility. …” (Kitab Futuh al-Buldan, vol.1, Pg 34) 
In another version, it is related to us that the Banu Nadir had asked Muhammed (p) to come out to them, planning to stab him up. But they didn’t succeed because a Jewess lady informed one of her own relatives who was on the Muslim side, on what the Banu Nadir were going to do (Ma’mar Ibn Rashid’s Kitab al-Maghazi).
After Muhammed (p) found out their plans, he asked the Banu Nadir to shun their treacherous plans against him, the Prophet (p) brought his hand forward, asking them to make a new treaty, the Banu Nadir categorically rejected peace. Instead, wanted to go war against the Muslims.
The infidels of the Quraysh again wrote (a letter) to the Jews after the battle of Badr: You are men of weapons and fortresses. You should fight our companion or we shall deal with you in a certain way. And nothing will come between us and the anklets of your women.
When their letter reached the Prophet, THEY GATHERED BANU AN-NADIR TO VIOLATE THE TREATY. They sent a message to the Prophet: Come out to us with thirty men from your companions, and thirty rabbis will come out from us till we meet at a central place where they will hear you. If they testify to you and believe in you, we shall believe in you.
The narrator then narrated the whole story. When the next day came, the Apostle of Allah went out in the morning with an army, and surrounded them. He told them: I swear by Allah, you will have no peace from me until you conclude a treaty with me. But THEY REFUSED TO CONCLUDE A TREATY WITH HIM. He therefore fought them the same day.
Next he attacked Banu Quraysh [Qurayzah] with an army in the morning, and left Banu an-Nadir. He asked them to sign a treaty and they signed it. He turned away from them and attacked Banu an-Nadir with an army. He fought with them until they agreed to expulsion. Banu an-Nadir were deported, and they took with them whatever their camels could carry, that is, their property, the doors of their houses, and their wood. … (Sunan Abi Dawud Book 19, Hadith 2998)
Kitab al-Maghazi – Ma’mar Ibn Rashid:
“Subsequently, the battle of Badr transpired. It was after the events at Badr that the infidel Quraysh wrote the Jews as follows: ‘Indeed, you are a well-armed and well-fortified people, so you had better kill our tribesman, or else we will surely take action and nothing will stand between us and the attendants of your womensfolk’ – by ‘the attendents of your womensfolk’ they meant their golden anklets.
When their letter reached the Jews, the clan of al-Nadir chose treachery. Then they sent a message to the Prophet, saying, ‘Come out to meet us with thirty of your companions, and we will come forth with thirty Rabbis. We can meet at such-and-such place, half-away between you and us, and listen to what you have to say. If the Rabbis believe in the truth of what you say and believe in you, then we shall all believe.’ The Prophet then set out, taking thirty of his companions with him.
Thirty of the Jews Rabbis also came out to meet him, and eventually they had all gathered at an open expanse of earth. The Jews began to say to one another, ‘How will we be able to reach Muhammad when he has thirty of his companions at his side – each of them more willing than the next to lay down his life for him?’
So they sent him this message: ‘How can we understand what’s being said if we number altogether sixty men? Come forward with only three of your companions, and three of our scholars will set out to meet with you so that they can listen to what you have to say. And if they believe in your message, we too will believe, all of us, and testify to the truth of your message.’ The Prophet then set out with only three of his companions.
The Jews had brought daggers and concealed them, for they wanted to assassinate God’s Messenger. However, an honest Jewess from al-Nadir sent word to her nephews – for her brother was one of the Muslim Allies – and she informed her brother about the plans of al-Nadir to betray the Messenger of God. Quickly her brother set off, and when he reached the Prophet he disclosed their secret plans before the Prophet had reached the Nadir clan.
The Prophet turned back and then came to the Nadir clan the next morning with several arrays of armed men and besieged them. He said to them, ‘Unless you enter into a pact with me, you’ll have no guarantee of protection.’ They refused to agree to a pact with the Prophet, so he and the Muslims fought against them that very day.
The following morning, the Prophet went to the Qurayzah clan with cavalry and several arrays of armed men, leaving the Nadir clan behind. He summoned the Qurayzah clan to make a pact with him, and so they did. The Prophet then turned away from the Qurayzah clan and headed back to the Nadir clan with his armed men.
He fought them and eventually they surrendered, agreeing to be exiled and to take with them only what their camels could carry, minus any arms – meaning weapons. The clan of al-Nadir left Medina carrying only as many of their effects as their camels could bear. These included even the doors of their homes and the wooden beams, for they had taken apart their houses and dismantled them to carry away all the wood they could salvage. …” (Kitab al-Maghazi – Ma’mar Ibn Rashid pg. 69 – 71) 
The Prophet (p) having found their plot that they were planning to assassinate him, and them categorically reject to sign a treaty of peace, the Muslims fought them until they were besieged in their forts. With the Muslims besieging their forts for two weeks. In the end, the Jewish tribe Banu Nadir, agreed to stop fighting and leave Madinah, taking with them whatever goods they could carry.
From the above historical evidences, we read that the Banu Nadir had friendly relations with the enemies of the Muslims. They deliberately handed over information to the Quraysh, with intend of killing Muslims. They were also guilty of having attempted to assassinate the Prophet (p). And their open refusal to renew the treaty was seen therefore tantamount to declaration of war. With these, the only course left for the Muslims was to treat them as enemies. The siege of the forts was laid, until they agreed to leave Madinah.
 Ibn Ishaq – Siratu Rasulillah – (Translator: A. Guillaume), page 437
 Kitab Al-tabaqat Al-Kabir by Ibn Sa’d – volume 2 – Page 69
 Kitab Futuh Al-Buldan, By al-Imam abu-l Abbas Ahmad Ibn-Jabir al-Baladhuri, (Translated by Philip Khuri Hitti, PH. D.), [New York – Columbia Univeristy, Longmans, Green & Co., Agents – London: P. S. King & Son, LTD. 1916], volume 1, page 34
 The Expeditions (‘Kitab al-Maghazi’) – An Early Biography Of Muhammad By Ma’mar Ibn Rashid [Translated: Sean W. Anthony – Copyright 2014 by New York University], page 69 – 71