The model will diverge at different epochs depending on the settings n_epochs

[Shaunixiayu]

Why I set the number of epochs to 100, when the model runs to the 100th epoch will diverge, its cycle loss all become very large, when the number of epochs is set to 50, it will diverge after 50 epochs, this is the reason for the learning rate decay, I have modified many times still can not solve this problem, I hope you can help me explain, thank you very much, the following is my training command: =========================
python train.py --dataroot./datasets/my_dataset --name test_digital_core --model cycle_gan -- batch_size 2 -- load_size 550 -- crop_size 500
--NetG resnet_9blocks -- num_threads 8 -- n_epochs 100 -- n_epochs_decay 100 -- display_freq 1000 -- save_latest_freq 10000 -- gpu_ids 0 -- input_nc 1 -- output_nc 1 -- lr 0.00005 -- lambda_identity 0.5 -- display_id 0 -- lr_policy linear -- lr_decay_iters 50

Here is a snippet of loss_log:
(Epoch: 100, iters: 450, time: 0.253, data: 0.003) D_A: 0.186 G_A: 0.381 cycle_A: 0.232 idt_A: 0.083 D_B: 0.156 G_B: 0.327 cycle_B: 0.279 idt_B: 0.052
(Epoch: 100, iters: 550, time: 1.679, data: 0.003) D_A: 0.208 G_A: 0.352 cycle_A: 0.216 idt_A: 0.080 D_B: 0.168 G_B: 0.453 cycle_B: 0.270 idt_B: 0.049
(Epoch: 101, iters: 100, time: 0.253, data: 0.309) D_A: 61.812 G_A: 108.169 cycle_A: 0.783 idt_A: 0.103 D_B: 86.232 G_B: 150.874 cycle_B: 1.823 idt_B: 0.156
(Epoch: 101, iters: 200, time: 0.248, data: 0.003) D_A: 1148.972 G_A: 1809.694 cycle_A: 2.853 idt_A: 0.198 D_B: 1108.889 G_B: 1652.918 cycle_B: 6.601 idt_B: 0.166
(Epoch: 101, iters: 300, time: 0.252, data: 0.007) D_A: 1772.589 G_A: 2515.135 cycle_A: 3.453 idt_A: 0.199 D_B: 1788.356 G_B: 2537.567 cycle_B: 6.494 idt_B: 0.154
(Epoch: 101, iters: 400, time: 0.253, data: 0.002) D_A: 2305.573 G_A: 3069.433 cycle_A: 3.639 idt_A: 0.174 D_B: 2186.480 G_B: 2944.759 cycle_B: 7.144 idt_B: 0.139
(Epoch: 101, iters: 500, time: 0.253, data: 0.002) D_A: 2599.390 G_A: 3456.715 cycle_A: 5.050 idt_A: 0.191 D_B: 2767.597 G_B: 3279.008 cycle_B: 6.658 idt_B: 0.180
(Epoch: 102, iters: 50, time: 0.253, data: 0.018) D_A: 3118.727 G_A: 4030.425 cycle_A: 4.975 idt_A: 0.196 D_B: 3209.157 G_B: 3905.536 cycle_B: 5.776 idt_B: 0.164

[junyanz]

Which version/code are you using? Have you tried training the model with our default dataset and see if it works?