https://optimization.cbe.cornell.edu/api.php?action=feedcontributions&feedformat=atom&user=SYSEN5800TAsCornell University Computational Optimization Open Textbook - Optimization Wiki - User contributions [en]2026-04-26T00:08:09ZUser contributionsMediaWiki 1.43.0https://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77512024 Cornell Optimization Open Textbook Feedback2024-12-17T23:26:55Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*Adding more details to the Application Section is recommended.<br />
*Please use FigureX as a reference in the text.<br />
*Avoid using contraction (e.g., there's) in scientific writing.<br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*Please label the figures with numbers and direct readers to the figures in text.<br />
*References were not well formatted (e.g., Yuan, Y. (2015b)).<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Eight step procedures]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Please use Latex equation editor for equations.<br />
*Please number and label all figures and tables, use FigureX, TableX as a reference in the text.<br />
*Please place references after the period at the end of each sentence and avoid after the optimization problems. This goes for all the sections in the wiki.<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*Most contents are from previous year. The Wiki page should be your original content.<br />
*It is also recommended to revise the citation format based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*The Numerical Example section is incomplete.<br />
*Application and Conclusion sections are missing.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*Please include more citations in Algorithm Discussion section to support the contents.<br />
*Please use FigureX as a reference in the text.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*There is an extra line of citation links in the References section.<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*More citations are needed for supporting your statement.<br />
<br />
== [[AdamW]] ==<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*More citations are needed for supporting the statement and applications.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Once abbreviations are introduced, please make sure the full term do not appear throughout the context.<br />
*Visualization section is not necessary. Figures should be embedded in the Wiki page.<br />
*Conclusion and References sections should not belong to Application section.<br />
*References are already shown at the bottom of the page. The extra list should be removed.</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77502024 Cornell Optimization Open Textbook Feedback2024-12-17T23:26:09Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*Adding more details to the Application Section is recommended.<br />
*Please use FigureX as a reference in the text.<br />
*Avoid using contraction (e.g., there's) in scientific writing.<br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*Please label the figures with numbers and direct readers to the figures in text.<br />
*References were not well formatted (e.g., Yuan, Y. (2015b)).<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Please use Latex equation editor for equations.<br />
*Please number and label all figures and tables, use FigureX, TableX as a reference in the text.<br />
*Please place references after the period at the end of each sentence and avoid after the optimization problems. This goes for all the sections in the wiki.<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*Most contents are from previous year. The Wiki page should be your original content.<br />
*It is also recommended to revise the citation format based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*The Numerical Example section is incomplete.<br />
*Application and Conclusion sections are missing.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*Please include more citations in Algorithm Discussion section to support the contents.<br />
*Please use FigureX as a reference in the text.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*There is an extra line of citation links in the References section.<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*More citations are needed for supporting your statement.<br />
<br />
== [[AdamW]] ==<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*More citations are needed for supporting the statement and applications.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Once abbreviations are introduced, please make sure the full term do not appear throughout the context.<br />
*Visualization section is not necessary. Figures should be embedded in the Wiki page.<br />
*Conclusion and References sections should not belong to Application section.<br />
*References are already shown at the bottom of the page. The extra list should be removed.</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77452024 Cornell Optimization Open Textbook Feedback2024-12-17T18:30:11Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*More citations are needed for supporting your statement.<br />
<br />
<br />
== [[AdamW]] ==<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*More citations are needed for supporting the statement and applications.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77442024 Cornell Optimization Open Textbook Feedback2024-12-17T18:28:21Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*More citations are needed for supporting your statement.<br />
<br />
<br />
== [[AdamW]] ==<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77432024 Cornell Optimization Open Textbook Feedback2024-12-17T18:26:37Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*More citations are needed for supporting your statement.<br />
<br />
<br />
== [[AdamW]] ==<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77422024 Cornell Optimization Open Textbook Feedback2024-12-17T18:23:28Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*More citations are needed for supporting your statement.<br />
<br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77412024 Cornell Optimization Open Textbook Feedback2024-12-17T18:21:25Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*More citations are needed for supporting your statement.<br />
<br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77402024 Cornell Optimization Open Textbook Feedback2024-12-17T18:17:35Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77392024 Cornell Optimization Open Textbook Feedback2024-12-17T18:15:01Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Citation form (in text) should be double checked.<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77382024 Cornell Optimization Open Textbook Feedback2024-12-17T18:12:34Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77372024 Cornell Optimization Open Textbook Feedback2024-12-17T18:08:03Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Form of in-text citation is not proper<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77362024 Cornell Optimization Open Textbook Feedback2024-12-17T18:01:31Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=77352024 Cornell Optimization Open Textbook Feedback2024-12-16T21:56:27Z<p>SYSEN5800TAs: /* Quadratic programming */</p>
<hr />
<div>== [[Computational complexity]] ==<br />
*References were not provided to support applications on computer science and quantum computing.<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*The title should be “Problem Solution” instead of ”Problem Resolution“.<br />
*The added figure is too small for viewing from the main page and also lacks captions and explanations.<br />
*There are still some formatting issues with the subtitles. For example, some are in bold form while others are not in the content.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*An inappropriate nonconvex example was used in the Wiki Page. The problem is an MIQP problem instead of a single QP problem.<br />
*Some symbols used in the pseudocodes still lack explanation.<br />
<br />
== [[Subgradient optimization]] ==<br />
*References were not well formatted.<br />
*Symbols used in equations lack explanations.<br />
*The application section was not well drafted and supported by references.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*There is a lack of numerical examples for illustrating the global optimization method introduced on the Wiki page.<br />
*The clarification of the equations used in the Introduction section needs to be improved.<br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
*The pseudocode was not well defined. More explanations and logical flow are needed.<br />
*Citations need to be included in the punctuation like period.<br />
*Since this is a modified version of Adam, a comparison with Adam is needed for the numerical example.<br />
*A machine learning case is needed since this is an algorithm designed for machine learning models.<br />
<br />
== [[FTRL algorithm]] ==<br />
*It is very difficult to clarify different levels of subtitles based on the text.<br />
*References were not well formatted. For example, "[1]" is added to many references with no meaning.<br />
*Some symbols used in the equation lack explanations.<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*The result figures of the numerical example have not been placed in the proper place.<br />
*The numerical example and application parts are still not representative of illustrating NDAM's performance for machine learning models.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*The caption was missed for the iteration results.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=FTRL_algorithm&diff=7734FTRL algorithm2024-12-16T21:11:45Z<p>SYSEN5800TAs: </p>
<hr />
<div>Author: Tsz Ki Peter Wei (tmw86), Nhi Nguyen (npn25), Colin Erb (cte24) (ChemE 6800 Fall 2024) <br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu<br><br />
<br />
== Introduction ==<br />
The FTRL (Follow the Regularized Leader) family of learning algorithms is a core set of learning methods used in online learning. As a type of FTL (Follow the Leader) algorithm, they select a weight function at each timestep that minimizes the loss of all previously observed data. To reduce computational complexity, implementations of the FTRL algorithm generally utilize a linearized loss function, while a regularizer ensures solution stability by limiting changes to the weight vector. The algorithm was originally introduced in 2008 by J.D. Abernethy, E. Hazan, and A. Rakhlin, with the key idea of “analysis through the dual” <ref name=":6">[1] bremen79, “Follow-The-Regularized-Leader I: Regret Equality,” Parameter-free Learning and Optimization Algorithms. Accessed: Dec. 03, 2024. [Online]. Available: <nowiki>https://parameterfree.com/2019/10/08/follow-the-regularized-leader-i-regret-equality</nowiki></ref> The use of linear losses in FTRL started in 2009 with the publishing of a paper by Y. Nesterov, although the idea is claimed to have originated in 2001-2002 but was deemed unimportant <ref name=":6" />. It is currently popular due to its asymptotically optimal regret and, in certain implementations, model sparsity <ref name=":0" /> in contrast to generic FTL algorithms, which have the dependency to overfit given a lack of regularization while minimizing total loss with the dimensionality available. A version of the algorithm using L1 and L2 regularizers was popularized by Google for ad-click advertising, specifically to predict ad click-through rates for sponsored search advertising<ref name=":0" />.<br />
<br />
== Algorithm Discussion ==<br />
<br />
==== '''General Algorithm Description:''' ====<br />
The FTRL algorithm is designed to minimize the objective function shown below. <br />
<br />
<math>w_{t+1} = \arg\min \left( l_{1:t}(w) + R(w) \right)<br />
<br />
</math> <br />
<br />
This minimization problem finds the weight vector minimizing the function <math>f(w) = l_{1:t}(w) + R(w)<br />
</math>, where <math> l_{1:t}(w) <br />
</math> represents the cumulative loss of all previous observations, and <math>R(w)<br />
</math> is a regularization term. Solving this objective, however, is extremely computationally expensive, as the loss must be expanded and recomputed for all past data points whenever new data is introduced. <br />
<br />
To address this, most implementations of the algorithm approximate the loss using a linearized loss function, leveraging the gradient of the original loss to reduce computational complexity. Unlike other machine learning algorithms that iteratively step along this gradient upon each update step, FTRL minimizes an optimization problem upon each update step, where instead the regularization places a limit on the step size. <br />
<br />
In order to evaluate the effectiveness of the model, the regret bound provides a strong metric for how well the model converges, by illustrating how similar previous predictions of the weight vector align to the optimal prediction generated using more data. FTRL has a regret bound of the following form: <br />
<br />
<math>\text{Regret}(u) \leq R(u) + \sum_{t=1}^T \left( l_t(w_t) - l_t(w_{t+1}) \right)<br />
</math> <br />
<br />
This indicates that the total regret is constrained by two factors, those being the regularization and cumulative loss from updating the weight vector with new data. This expression is derived from the below relation<ref name=":7" />. <br />
<br />
<math>\sum_{t=0}^T \left( f_t(w_t) - f_t(u) \right) \leq \sum_{t=0}^T \left( f_t(w_t) - f_t(w_{t+1}) \right)<br />
<br />
</math> <br />
<br />
This lemma holds true, as at best <math>f_t(w_{t+1})<br />
<br />
<br />
</math> is equal to <math>f_t(u)<br />
<br />
</math> for all run times. In practice, the presence of the regularization term prevents <math> f_t(w_t) <br />
<br />
</math> from matching the optimal solution, but it helps minimize regret in more noisy environments, where the decision variable between steps varies significantly without it. <br />
<br />
==== '''Common Implementations:''' ====<br />
<br />
===== '''Linear Loss FTRL''' =====<br />
One of the simplest forms of the FTRL algorithm utilizes a linearized loss function, reducing the cost of recomputing all past losses, and an L<sub>2</sub> regularization term to stabilize the solution. As such, the weight update is defined as follows <ref name=":7">[1] B. McMahan, “Follow-the-regularized-leader,” CSE599s, <nowiki>https://courses.cs.washington.edu/courses/cse599s/14sp/scribes/lecture3/lecture3.pdf</nowiki> (accessed Dec. 14, 2024).</ref>: <br />
<br />
<math>w_{t+1} = \arg\min_{w \in \mathbb{R}^n} \left( g_{1:t} \cdot w + \frac{1}{2\eta} \|w\|_2^2 \right)<br />
<br />
<br />
</math>Minimizing this expression involves setting the gradient with respect to weight vector to 0: <br />
<br />
<math>\nabla_w \left( g_{1:t} \cdot w + \frac{1}{2\eta} \|w\|_2^2 \right) = g_{1:t} + \frac{1}{\eta} w = 0 \implies w = -\eta g_{1:t}<br />
<br />
</math> <br />
<br />
In the update step, the gradient of the loss function, <math> g_{1:t}<br />
<br />
<br />
<br />
</math> can be approximated as <math>g_{1:t} = -\eta \left( g_t + \frac{-w_t}{\eta} \right)<br />
<br />
<br />
<br />
</math>, based on the solution to the previous iteration producing a solution for w. Utilizing this, the weight vector update for each new data point becomes <math>w_{t+1} = w_t - \eta g_t<br />
<br />
<br />
<br />
</math>. <br />
<br />
This is the same formulation as that of gradient descent, making this a special case of the FTRL algorithm <ref name=":7" />, and shows an important parallel between the formulations and how their processing of data compares, despite the different problem formulations. <br />
<br />
====== Model Pseudo-code: ======<br />
Initialize parameters: <br />
<br />
s = 0 <br />
<br />
z = 0 <br />
<br />
w=0 <br />
<br />
for each iteration t= 1 ... T: <br />
<br />
# (x<sub>t</sub>, y<sub>t</sub>): Receive a new data point.<br />
# (<math>\hat{y}_t = \mathbf{w}^\top \mathbf{x}_t<br />
</math>): Compute the predicted value.<br />
# (<math>g_{1:t} = -\eta \left( g_t + \frac{\mathbf{w}_t}{\eta} \right)<br />
</math>): Compute the accumulating gradient of the loss function L.<br />
# (<math>\mathbf{w}_{t+1} = -\eta g_{1:t}<br />
</math>):Update the weight vector.<br />
<br />
===== '''FTRL-Proximal algorithm''' =====<br />
Some kinds of regularization terms, such as that used in ad-click advertising by Google, combine both L1 and L2 loss to produce solutions that are both stable and sparse, similarly to other online gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD). An instance of the FTRL algorithm using both L1 and L2 regularization terms is shown below. <br />
<br />
<math>w = \arg\min \left( g_t^\top w + \sum_{s=1}^t \frac{1}{2\eta_s} \|w - w_s\|^2 + t\lambda \|w\|_1 \right)<br />
<br />
<br />
</math> <br />
<br />
This version of the FTRL algorithm is particularly adept at introducing sparsity, given the t copies of the L1 regularization term introduced for each time step in the online update process <ref name=":8">[1] B. McMahan, “The FTRL algorithm with strongly convex Regularizers,” cse599s, <nowiki>https://courses.cs.washington.edu/courses/cse599s/12sp/scribes/Lecture8.pdf</nowiki> (accessed Dec. 14, 2024).</ref>. It promotes model stability as the dataset size grows via the second term in the model, which decreases each weight’s update size proportionally to its update history. The solution for the weight vector update is shown below, with i representing each dimension<ref name=":0" />. <br />
<br />
<math>w_{t+1,i} =<br />
\begin{cases} <br />
-\eta_{t} \left( z_{t,i} - \text{sign}(z_{t,i}) \lambda_1 \right), & \text{if } |z_{t,i}| \leq \lambda_1 \\<br />
0 & \text{otherwise}<br />
\end{cases}<br />
<br />
<br />
<br />
</math><br />
<br />
<math>z_t = g_{1:t-1} - \sum_{s=1}^{t-1} \sigma_s w_s + g_t + \left( \frac{1}{\eta_t} - \frac{1}{\eta _{t-1}} \right) w_t<br />
<br />
<br />
<br />
<br />
</math>.<br />
<br />
Typically, the z value’s historical data would be saved<ref name=":0" />, thus making the update step for z<sub>t</sub> as follows: <br />
<br />
<math>z_t = z_{t-1} + g_t + \left( \frac{1}{\eta _t} - \frac{1}{\eta _{t-1}} \right) w_t<br />
<br />
<br />
<br />
</math><br />
<br />
The learning rate is set dynamically using the following relation, where α and β are hyper-parameters set ahead of time to maximize performance:<br />
<br />
<math>\eta_{t, i} = \frac{\alpha}{\beta + \sqrt{\sum_{s=1}^t g_{s, i}^2}}<br />
<br />
<br />
<br />
<br />
</math><br />
<br />
There are a few major consequences of this algorithm. The consideration of the L1 regularization and L2 regularization through stabilization encouraging sparse solutions where weights less than lambda are pulled to 0, and exceedingly large gradients across previous iterations are updated less, as to encourage more unstable dimensions to stabilize and allow stable dimensions room to explore<ref name=":8" />. As such, this model balances stability and sparsity to form a highly versatile model. <br />
<br />
====== Model Pseudocode: ======<br />
Initialize parameters: <br />
<br />
s = 0 <br />
<br />
z = 0 <br />
<br />
w=0 <br />
<br />
for each iteration t= 1 ... T: <br />
<br />
# (x<sub>t</sub>, y<sub>t</sub>): Receive a new data point.<br />
# (<math>\hat{y}_t = \mathbf{w}^\top \mathbf{x}_t<br />
</math>): Compute the predicted value.<br />
# (<math>g_{t, i} = \frac{\partial L(\hat{y}_t, y_t)}{\partial \mathbf{w}}</math>): Compute the gradient of the loss function L with respect to w.<br />
# (<math>\mathbf{s_i} = \mathbf{s_i} +(g_i)^2<br />
</math>): Update the accumulated squared gradients.<br />
# (<math>\eta_i = \frac{\alpha}{\sqrt{s_i} + \beta}<br />
<br />
</math>): Update the learning rate for each feature i.<br />
# (<math>z_{t, i} = z_{t-1, i} + g_{t, i} + \left( \frac{1}{\eta_t} - \frac{1}{\eta_{t-1}} \right) w_{t, i}<br />
</math>): Update the z-values for each feature i.<br />
# (<math>w_{t+1, i} =<br />
\begin{cases} <br />
-\eta_{t, i} \left( z_{t, i} - \text{sign}(z_{t, i}) \lambda_1 \right) & \text{if } |z_{t, i}| > \lambda_1 \\<br />
0 & \text{otherwise}<br />
\end{cases}<br />
</math>): Update the weight vector<br />
<br />
== Numerical Examples ==<br />
=== Example 1: Linear Loss with Quadratic Regularization ===<br />
<br />
'''Problem Setup:'''<br />
* Decision variable: <math>w \in \mathbb{R}</math>,<br />
* Loss functions: <math>f_t(w) = a_t w</math>, where <math>a_t</math> are coefficients provided by the environment. In the case of a linear loss function, <math>a_t</math> determines the slope or gradient of the loss function <math>f_t(w)</math>,<br />
* Regularization: <math>R(w) = \frac{\lambda}{2} w^2</math>, where <math>\lambda</math> is a hyperparameter that determines the update rate.<br />
<br />
'''FTRL Update Rule:'''<br />
The update rule is:<br />
<math><br />
w_{t+1} = \arg\min_w \left( \sum_{i=1}^t a_i w + \frac{\lambda}{2} w^2 \right).<br />
</math><br />
<br />
To minimize, take the derivative and set it to zero:<br />
<math><br />
\frac{\partial}{\partial w} \left( \sum_{i=1}^t a_i w + \frac{\lambda}{2} w^2 \right) = \sum_{i=1}^t a_i + \lambda w = 0.<br />
</math><br />
<br />
Rearranging gives:<br />
<math><br />
w_{t+1} = -\frac{\sum_{i=1}^t a_i}{\lambda}.<br />
</math><br />
<br />
'''Step-by-Step Calculation:'''<br />
Let the gradient of the loss functions be <math>a_1 = -1</math>, <math>a_2 = 1.75</math>, <math>a_3 = -1.95</math>, initialize <math>w_0 = 0</math>, and choose <math>\lambda = 3</math>.<br />
<br />
* Iteration 1:<br />
<math><br />
w_1 = -\frac{a_1}{\lambda} = -\frac{-1}{3} = 0.33.<br />
</math><br />
<br />
* Iteration 2:<br />
<math><br />
w_2 = -\frac{a_1 + a_2}{\lambda} = -\frac{(-1) + 1.75}{3} = -0.25.<br />
</math><br />
<br />
* Iteration 3:[[File:FTRL_Linear_Loss_Stabilization_Corrected.png|thumb|450px|alt=Stabilizing effects of FTRL for linear loss function|'''Fig 1.''' Stabilizing effects of FTRL for linear loss function. <br> Without regularization (blue), updates fluctuate significantly. With regularization (orange), updates stabilize over tim]]<br />
<math><br />
w_3 = -\frac{a_1 + a_2 + a_3}{\lambda} = -\frac{(-1) + 1.75 + (-1.95)}{3} = 0.4.<br />
</math><br />
<br />
'''Observation:'''<br />
The updates for <math>w</math> gradually converge rather than overshooting or oscillating, thanks to regularization reducing large shifts with the term <math>\lambda</math> in the update rule. Decision variables calculated with and without the regularization term in the update rule (i.e. setting <math>\lambda=1</math>) are compared for 10 iterations to demonstrate this stabilizing effect.<br />
* Without Regularization: Updates (<math>w_t</math>) exhibit significant fluctuations as they respond solely to the cumulative gradients <math>\sum_{i=1}^t a_i</math>.<br />
* With Regularization: Updates are smoother, as the regularization term <math>\lambda</math> reduces the influence of cumulative gradients, stabilizing the updates over time.<br />
<br />
=== Example 2: Quadratic Loss with Quadratic Regularization ===<br />
<br />
'''Problem Setup:'''<br />
* Quadratic loss function: <math>f_t(w) = \frac{1}{2} (w - b_t)^2</math>, where <math>b_t</math> is the target set by the adversary,<br />
* Regularization: <math>R(w) = \frac{\lambda}{2} w^2</math>.<br />
<br />
'''FTRL Update Rule:'''<br />
The update rule becomes:<br />
<math><br />
w_{t+1} = \arg\min_w \left( \sum_{i=1}^t \frac{1}{2} (w - b_i)^2 + \frac{\lambda}{2} w^2 \right).<br />
</math><br />
<br />
Simplifying:<br />
<math><br />
w_{t+1} = \frac{\sum_{i=1}^t b_i}{t + \lambda}.<br />
</math><br />
<br />
'''Step-by-Step Calculation:'''<br />
Let coefficients <math>b_1 = 1</math>, <math>b_2 = 2</math>, <math>b_3 = 3</math>, initialize <math>w_0 = 0</math>, and choose <math>\lambda = 2</math>.<br />
<br />
* Iteration 1:<br />
<math><br />
w_1 = \frac{b_1}{1 + \lambda} = \frac{1}{1 + 2} = 0.33.<br />
</math><br />
<br />
* Iteration 2:<br />
<math><br />
w_2 = \frac{b_1 + b_2}{2 + \lambda} = \frac{1 + 2}{2 + 2} = 0.75.<br />
</math><br />
<br />
* Iteration 3:[[File:FTRL_Quadratic_Loss_Stabilization.png|thumb|450px|alt=Stabilizing effects of FTRL for quadratic loss function|'''Fig 2.''' Stabilizing effects of FTRL for quadratic loss function. <br> Without regularization (blue), updates grow unbounded. With regularization (orange), updates converge gradually.]]<br />
<math><br />
w_3 = \frac{b_1 + b_2 + b_3}{3 + \lambda} = \frac{1 + 2 + 3}{3 + 2} = 1.2.<br />
</math><br />
<br />
'''Observation:'''<br />
The denominator <math>t + \lambda</math> ensures that updates become smaller over time, stabilizing the solution as more data is observed. Decision variables calculated with and without the regularization term are compared for 10 iterations to demonstrate this stabilizing effect in Figure 2.<br />
<br />
* Without Regularization: Linear and unbounded growth. Updates (<math>w_t</math>) grow rapidly as they are driven entirely by the cumulative target values <math>\sum_{i=1}^t b_i</math>, leading to unbounded growth over time.<br />
* With Regularization: Gradual convergence with stabilized updates. The denominator <math>t + \lambda</math> dampens the influence of cumulative targets, stabilizing the updates.<br />
<div style="clear: both;"></div><br />
<br />
== Application ==<br />
The FTRL algorithm is a powerful framework for online learning problems due to its ability to handle large datasets and adapt to new data in real-time. It has been applied in finance, healthcare, and electrical engineering. In finance, the FTRL algorithm has been used in online advertising, e-commerce recommendations, and fraud detection <ref name=":0">H. B. McMahan ''et al.'', “Ad click prediction: a view from the trenches,” in ''Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining'', Chicago Illinois USA: ACM, Aug. 2013, pp. 1222–1230. doi: 10.1145/2487575.2488200.</ref><ref name=":1">J. O. Schneppat, “Follow The Regularized Leader (FTRL),” Schneppat AI. Accessed: Nov. 30, 2024. [Online]. Available: <nowiki>https://schneppat.com/follow-the-regularized-leader_ftrl.html</nowiki></ref>. In healthcare, the FTRL algorithm has been used for the analysis of patient data <ref>Z. Ye, F. Chen, and Y. Jiang, “Analysis and Privacy Protection of Healthcare Data Using Digital Signature,” in ''Proceedings of the 2024 3rd International Conference on Cryptography, Network Security and Communication Technology'', Harbin China: ACM, Jan. 2024, pp. 171–176. doi: 10.1145/3673277.3673307.</ref>. In electrical engineering, the FTRL algorithm has been used to optimize the performance of electrical power systems <ref name=":2">X. Gao ''et al.'', “Followed The Regularized Leader (FTRL) prediction model based photovoltaic array reconfiguration for mitigation of mismatch losses in partial shading condition,” ''IET Renew. Power Gener.'', vol. 16, no. 1, pp. 159–176, 2022, doi: 10.1049/rpg2.12275.</ref><ref name=":3">C. Gao, Z. Ding, S. Yan, and H. Mai, “Low Voltage Prediction Based on Spark and Ftrl,” in ''Proceedings of the Advances in Materials, Machinery, Electrical Engineering (AMMEE 2017)'', Tianjin City, China: Atlantis Press, 2017. doi: 10.2991/ammee-17.2017.34.</ref>. Specific case studies of the FTRL algorithm in each field are shown below. <br />
<br />
=== Finance ===<br />
The first application that the FTRL algorithm was used for was online advertising by Google <ref name=":0" />. In this case study, the FTRL algorithm was used to predict ad click-through rates (CTR) for sponsored search advertising at Google <ref name=":0" />. They were able to demonstrate the FTRL algorithm’s ability to both predict accurately and also be sparse compared to other online learning algorithms <ref name=":0" />. Additional case studies in the finance sector have been explored such as online portfolio optimization <ref name=":4">R. Jézéquel, D. M. Ostrovskii, and P. Gaillard, “Efficient and Near-Optimal Online Portfolio Selection,” Sep. 28, 2022, ''arXiv'': arXiv:2209.13932. doi: 10.48550/arXiv.2209.13932.</ref>. In this case study, the authors were able to build off the original FTRL algorithm to come up with a new algorithm called VB-FTRL that can maximize a trader’s return on their portfolio while having reduced runtime compared to the best-performing algorithm (Universal Portfolios) <ref name=":4" />. <br />
<br />
=== Healthcare ===<br />
In healthcare, the FTRL algorithm was used in a case study to classify thyroid nodules for Thyroid cancer diagnosis <ref name=":5">A. Beyyala, R. Priya, S. R. Choudari, and R. Bhavani, “Classification of Thyroid Nodules Using Follow the Regularized Leader Optimization Based Deep Neural Networks. | EBSCOhost.” Accessed: Nov. 30, 2024. [Online]. Available: <nowiki>https://openurl.ebsco.com/contentitem/doi:10.18280%2Fria.370315?sid=ebsco:plink:crawler&id=ebsco:doi:10.18280%2Fria.370315</nowiki></ref>. In this case study, the authors proposed a novel FTRL-Deep Neural Network technique to precisely classify thyroid nodules into benign or malignant. The algorithm would analyze ultrasound images of the thyroid nodules and then determine whether or not the patient has thyroid cancer or not <ref name=":5" />. They were able to show that their FTRL-Deep Neural Network algorithm had superior accuracy compared to other algorithms such as the Hybrid Feature Cropping Network and Multi-Channel Convolutional Neural Network <ref name=":5" />.<br />
<br />
=== Electrical Engineering ===<br />
Recently, FTRL has been applied to electrical power systems. A recent case study used FTRL to optimize the output of a solar panel system to offset the effect of non-uniform irradiance <ref name=":2" />. They developed this algorithm to control a system of switches that would determine the optimal configuration of the solar panel array when non-uniform irradiance is detected to maximize the power output from the solar panel <ref name=":2" />. In an earlier case study, researchers were able to use an FTRL algorithm to predict when low voltages would occur in a power distribution system, allowing for better system management <ref name=":3" />. <br />
<br />
=== Softwares and Platforms that Utilize the FTRL Algorithm ===<br />
Software tools such as H2O Driverless AI and Keras can utilize the FTRL algorithm for various machine-learning applications <ref>“Supported Algorithms — Using Driverless AI 1.11.0 documentation.” Accessed: Nov. 30, 2024. [Online]. Available: <nowiki>https://docs.h2o.ai/driverless-ai/1-10-lts/docs/userguide/supported-algorithms.html</nowiki></ref><ref>K. Team, “Keras documentation: Ftrl.” Accessed: Nov. 30, 2024. [Online]. Available: <nowiki>https://keras.io/api/optimizers/ftrl/</nowiki></ref>. In addition, platforms such as Google Ads utilize the FTRL algorithm <ref name=":1" />. These are just a few examples of softwares and platforms where the FTRL algorithm is used. As machine-learning becomes more and more widespread in different applications, the usage of the FTRL algorithm is expected to increase. <br />
<br />
== Conclusion ==<br />
The FTRL algorithm is most useful in online learning, where large amounts of data need to be processed in real time to enable accurate predictions. Currently, the FTRL algorithm is used on platforms such as Google Ads and included in software tools such as Keras and H2O driverless AI. In recent years, other areas of application that involve online learning, such as healthcare and power systems, have been explored for the FTRL algorithm.<br />
<br />
The algorithm works through a minimization of a linearized loss function considering all historical data and at least one regularization term every time a new point is added. This minimization operation is reasonably space and time-efficient due to linearization, and due to flexibility in how the weight vector is regularized, very versatile. The simplest implementation of the algorithm is equivalent to gradient descent, while more complex versions of the algorithm are able to effectively stabilize the weight vector as the dataset grows and maintain sparsity in its updates through the inclusion of an L1 norm.<br />
<br />
From the numerical examples, it is evident that regularization plays a pivotal role in ensuring convergence. For linear loss functions, FTRL mitigates oscillatory behavior by reducing the influence of cumulative gradients. For quadratic loss functions, regularization stabilizes updates over time, resulting in gradual convergence and bounded growth. These properties underline FTRL's adaptability in noisy environments and its capacity to produce reliable predictions.<br />
<br />
Looking ahead, potential areas for improvement include enhancing computational efficiency for more complex loss functions and exploring hybrid regularization schemes that combine the strengths of L1, L2, and other techniques. Additionally, further advancements in applying FTRL to evolving fields such as autonomous systems, environmental modeling, and personalized medicine could broaden its impact.<br />
<br />
== References ==<br />
<references /><br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71972024 Cornell Optimization Open Textbook Feedback2024-12-13T18:53:19Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*The contents are good, but need to improve the clarity of the Wiki page.<br />
*The citations need to be numbered by the order of their appearance.<br />
*Provide more details on the Global Optimization Techniques and Metaheuristics approach that you mentioned.<br />
*Please provide step-by-step calculation processes for the numerical examples.<br />
*Please check some minor grammer issues in the file.<br />
*For applications, please try to use coherent paragraphs instead of the lists of bullet items.<br />
*1, 2, and n in "x = [x1, x2, ..., xn]" should be subscripted in the Introduction section. <br />
*"Minimize" and "Subject to" should not be italicized in equations. <br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.<br />
*If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".<br />
*Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?<br />
*For introduction, no need to give so many words on introducing other methods.<br />
*Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71942024 Cornell Optimization Open Textbook Feedback2024-12-13T18:51:55Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
*It would be better to have subsections of each approach in the Algorithm Discussion section to improve the clarity.<br />
*The logic flow and clarity of both the Algorithm Discussion section and Numerical Example section need to be largely improved.<br />
*Please include citations to support the Wiki contents.<br />
*Please make sure abbreviations should be consistent throughout the context.<br />
*In-text citations are required.<br />
*Please add mathematical expressions to Algorithm Discussion section for explicitness.<br />
*Please provide step by step calculation for the numerical example.<br />
*Adding some figures may help reader to understand the content.<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*The contents are good, but need to improve the clarity of the Wiki page.<br />
*The citations need to be numbered by the order of their appearance.<br />
*Provide more details on the Global Optimization Techniques and Metaheuristics approach that you mentioned.<br />
*Please provide step-by-step calculation processes for the numerical examples.<br />
*Please check some minor grammer issues in the file.<br />
*For applications, please try to use coherent paragraphs instead of the lists of bullet items.<br />
*1, 2, and n in "x = [x1, x2, ..., xn]" should be subscripted in the Introduction section. <br />
*"Minimize" and "Subject to" should not be italicized in equations. <br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.<br />
*If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".<br />
*Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?<br />
*For introduction, no need to give so many words on introducing other methods.<br />
*Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==<br />
*Provide more references to support the statements made in the Application Section.<br />
*Make sure abbreviations are consistent throughout the context (e.g. Please define DE at the introduction section not in the later section, please check the similar cases as well).<br />
*In-text citations are required.<br />
*The symbols in the pseudocode is not in proper format. Please use Latex equation editor for this.<br />
*Please number and label all figures, use FigureX as a reference in the text.<br />
*Please add more details to numerical example (e.g., replace x = -7 with -5 because f(x1) > f(u1))</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71912024 Cornell Optimization Open Textbook Feedback2024-12-13T18:48:21Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
*More details on the Application Section.<br />
*Please come up with one numerical example with hill-climbing.<br />
*Please add more references to support the content in the Introduction section.<br />
*Please add figure captions for the pseudocodes. <br />
*Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.<br />
*ResearchGate is not a publisher. Please check the reference again. <br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*The contents are good, but need to improve the clarity of the Wiki page.<br />
*The citations need to be numbered by the order of their appearance.<br />
*Provide more details on the Global Optimization Techniques and Metaheuristics approach that you mentioned.<br />
*Please provide step-by-step calculation processes for the numerical examples.<br />
*Please check some minor grammer issues in the file.<br />
*For applications, please try to use coherent paragraphs instead of the lists of bullet items.<br />
*1, 2, and n in "x = [x1, x2, ..., xn]" should be subscripted in the Introduction section. <br />
*"Minimize" and "Subject to" should not be italicized in equations. <br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.<br />
*If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".<br />
*Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?<br />
*For introduction, no need to give so many words on introducing other methods.<br />
*Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71892024 Cornell Optimization Open Textbook Feedback2024-12-13T18:46:04Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
*For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.<br />
*Attaching code to the Wiki editing page is not recommended.<br />
*It would be better to provide more references for supporting the statements made in the Application Section.<br />
*Please try to add more citations in the introduction section.<br />
*In-text citations are required.<br />
*Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.<br />
*Please add some mathematical expressions to Algorithm Discussion section for explicitness. <br />
*Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.<br />
*In-text citation should be added to figure captions. <br />
<br />
<br />
<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
*If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.<br />
*It is also recommended to revise the citation format in the file based on our example files.<br />
*Please add the case of non convex functions.<br />
*Please add references to support the content in the Introduction section. In-text citations are required.<br />
*Please use Latex equation editor for typing symbols and equations.<br />
*There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details. <br />
*The provided example is somewhat trivial. Please provide a more sophisticated example.<br />
*It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.<br />
<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*The contents are good, but need to improve the clarity of the Wiki page.<br />
*The citations need to be numbered by the order of their appearance.<br />
*Provide more details on the Global Optimization Techniques and Metaheuristics approach that you mentioned.<br />
*Please provide step-by-step calculation processes for the numerical examples.<br />
*Please check some minor grammer issues in the file.<br />
*For applications, please try to use coherent paragraphs instead of the lists of bullet items.<br />
*1, 2, and n in "x = [x1, x2, ..., xn]" should be subscripted in the Introduction section. <br />
*"Minimize" and "Subject to" should not be italicized in equations. <br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.<br />
*If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".<br />
*Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?<br />
*For introduction, no need to give so many words on introducing other methods.<br />
*Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71872024 Cornell Optimization Open Textbook Feedback2024-12-13T18:42:37Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*The contents are good, but need to improve the clarity of the Wiki page.<br />
*The citations need to be numbered by the order of their appearance.<br />
*Provide more details on the Global Optimization Techniques and Metaheuristics approach that you mentioned.<br />
*Please provide step-by-step calculation processes for the numerical examples.<br />
*Please check some minor grammer issues in the file.<br />
*For applications, please try to use coherent paragraphs instead of the lists of bullet items.<br />
*1, 2, and n in "x = [x1, x2, ..., xn]" should be subscripted in the Introduction section. <br />
*"Minimize" and "Subject to" should not be italicized in equations. <br />
<br />
== [[Adadelta]] ==<br />
*Adding a subsection title to divide the numerical examples will improve the clarity of this file.<br />
*Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.<br />
*Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well." <br />
*Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.<br />
*If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".<br />
*Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?<br />
*For introduction, no need to give so many words on introducing other methods.<br />
*Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71862024 Cornell Optimization Open Textbook Feedback2024-12-13T18:41:09Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
*The contents is good to improve the clarity of the Wiki page.<br />
*The citations need to be numbered by the order of their appearance.<br />
*Provide more details on the Global Optimization Techniques and Metaheuristics approach that you mentioned.<br />
*Please provide step-by-step calculation processes for the numerical examples.<br />
*Please check some minor grammer issues in the file.<br />
*For applications, please try to use coherent paragraphs instead of the lists of bullet items.<br />
*1, 2, and n in "x = [x1, x2, ..., xn]" should be subscripted in the Introduction section. <br />
*"Minimize" and "Subject to" should not be italicized in equations. <br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.<br />
*If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".<br />
*Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?<br />
*For introduction, no need to give so many words on introducing other methods.<br />
*Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71852024 Cornell Optimization Open Textbook Feedback2024-12-13T18:40:02Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
*Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.<br />
*If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".<br />
*Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?<br />
*For introduction, no need to give so many words on introducing other methods.<br />
*Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71842024 Cornell Optimization Open Textbook Feedback2024-12-13T18:38:21Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
*"Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.<br />
*"While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.<br />
*Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"<br />
*Clear explanations are needed to present the meaning of all parameters/variables used in equations.<br />
*The table would be better by being transposed.<br />
*Please provide more references to support the statements in the Application Section.<br />
*Please include the pseudocode for your algorithm.<br />
*Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.<br />
*Wikipedia is not a proper citation source. Please avoid citing Wikipedia.<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71832024 Cornell Optimization Open Textbook Feedback2024-12-13T18:36:51Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
*"The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.<br />
*It should be "Google" instead of "google".<br />
*It should be "data point" instead of "datapoint".<br />
*Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.<br />
*Please try to improve the clarity and logic flow of the algorithm description part.<br />
*"Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.<br />
*"The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"<br />
*It is suggested to include pseudocode for this algorithm.<br />
*Some equations are shown in figure form. Please use Latex equation editor for equations. <br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71812024 Cornell Optimization Open Textbook Feedback2024-12-13T18:32:44Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
*"The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?<br />
*"Relevance to modern optimization problems makes PSO an interesting research area."<br />
*What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.<br />
*"The convergence of the algorithm can be checked in a few different ways"<br />
*It should be "hyperparameters" instead of "hyper parameters"<br />
*It should be "Python" instead of "python"<br />
*It may not be good to provide code in the Wiki editing page.<br />
*In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.<br />
*"Depending on how the code is set up, scaling and offset factors may be required."<br />
*Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.<br />
*Please revise the reference formate based on our provided examples.<br />
*Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box. <br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71802024 Cornell Optimization Open Textbook Feedback2024-12-13T18:30:41Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
*If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".<br />
*Please explain every variable/parameters used in the equations.<br />
*It may be good to generate a table or figure to summarize the results of the numerical examples.<br />
*Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.<br />
*It would be better to provide more references to support the statements in the Application Section.<br />
*Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.<br />
*Please try to come up with flowchart/pseudocode for demonstrating the steps.<br />
*Please replace Step 1, 2, 3 with correct subheadings in Application section.<br />
*Websites and Youtube often are not proper citation sources.<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71782024 Cornell Optimization Open Textbook Feedback2024-12-13T18:29:06Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
*The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71742024 Cornell Optimization Open Textbook Feedback2024-12-13T18:11:24Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*"Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.<br />
*Put the citation number within the punctuation. Please revise this problem for the rest of the file.<br />
*Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.<br />
*"The smaller the value of this function, the higher the efficiency of the algorithm. "<br />
*The example 1 has some grammar issue.<br />
*This is topic-specific suggestion:<br />
*Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.<br />
*When stating potential applications, please provide related references.<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71732024 Cornell Optimization Open Textbook Feedback2024-12-13T18:09:37Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
*"Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "<br />
*"For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."<br />
*"Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "<br />
*it would be better to provide definitions of all parameters/variables used in the pseudocode.<br />
*same pseudocode problem<br />
*"Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.<br />
*Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.<br />
*In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.<br />
*"Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"<br />
*"This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.<br />
*"An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.<br />
*"GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"<br />
*check the grammer of the conclusion texts.<br />
*Please provide one example of non convex QP problem.<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71722024 Cornell Optimization Open Textbook Feedback2024-12-13T18:06:56Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
*Provide more citations to support your statements in the Application section?<br />
*Provide a figure to demonstrate the results generated from the numerical example?<br />
*Please provide more applications<br />
*Please provide the platform which used such algorithms<br />
*The Conclusion section is missing.<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71712024 Cornell Optimization Open Textbook Feedback2024-12-13T18:05:22Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
*Please double-check if the citations are correctly formatted in the text.<br />
*For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.<br />
*Please provide flowchart/pseudocode for representing your procedure<br />
*Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)<br />
*Avoid using pronouns (e.g., we, let's) in scientific writing.<br />
*The code snippet in the Application section can be removed.<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71702024 Cornell Optimization Open Textbook Feedback2024-12-13T18:04:09Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
*Please avoid the use of the term like "you" or "we" in the Wiki file.<br />
*Please include more citations to support your statements in the application section.<br />
*Please add more citations to support your statement in the introduction section<br />
*Please try to include a flowchart or pseudocode for illustration of the algorithm.<br />
*If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).<br />
*Please add subheadings in the Application section for better readability.<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71692024 Cornell Optimization Open Textbook Feedback2024-12-13T18:02:42Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
*The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.<br />
*Please revise your reference format.<br />
*More citations are needed for supporting your statement in the introduction section.<br />
*For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).<br />
*The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.<br />
*Algorithm and theory section is missing. <br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71682024 Cornell Optimization Open Textbook Feedback2024-12-13T18:01:25Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
*Please make citations immediately after the citing contents. Please double-check all the citations in your file.<br />
*Please include more citations in the application section to support your statements.<br />
*Please use flowchart/pseudocode for representing the steps of algorithm<br />
*Check the consistency of abbreviations (e.g. what is PAWS?)<br />
*Sections 5-7 can be combined into one section and divided by subsections.<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71672024 Cornell Optimization Open Textbook Feedback2024-12-13T18:00:00Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
*Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).<br />
*Citation number should be double checked.<br />
*You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*There is no need to show your own code in this wiki page.<br />
*Avoid adding citations in the conclusion section<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71662024 Cornell Optimization Open Textbook Feedback2024-12-13T17:58:43Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
<br />
== [[Genetic algorithm]] ==<br />
*Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.<br />
*Please provide some citations for supporting your statement (e.g. in Introduction)<br />
*Abbreviations should be consistent throughout the context (e.g. GA)<br />
*Please avoid citing or adding links to Wikipedia.<br />
*Please remove the sentence, "as documented in Computational Optimization and Applications."<br />
*More details are expected for Algorithm Discussion section.<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71652024 Cornell Optimization Open Textbook Feedback2024-12-13T17:56:38Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
*Please double-check that all cited Figures are clearly attached with citations.<br />
*Please double-check the citations of your references.<br />
*Need significant amount of citation to support the statement<br />
*Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.<br />
*Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)<br />
*Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)<br />
*Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
*Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71642024 Cornell Optimization Open Textbook Feedback2024-12-13T17:54:50Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
== [[Simulated annealing]] ==<br />
*Please provide citations in the application section to support your statements.<br />
*Provide some figures for the numerical study<br />
*Substitute those symbols in a formal way (e.g. T_min should be in a formal way).<br />
*Once the abbr. is defined please use it throughout the context (e.g. SA)<br />
*There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)<br />
*Avoid using pronouns (e.g., we) in scientific writing.<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71632024 Cornell Optimization Open Textbook Feedback2024-12-13T17:53:35Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
== [[Simulated annealing]] ==<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71622024 Cornell Optimization Open Textbook Feedback2024-12-13T17:53:01Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
*For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.<br />
*Please try to update equations on page 5. <br />
*Please try to use the pseudocode for the procedure described on page 3.<br />
*m_k and p in eq.(1) are not defined.<br />
*Fix the typo “prediced reduction” in eq.(3).<br />
*Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.<br />
*Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.<br />
*The termination conditions are not complete in section “Termination Criteria”.<br />
*Fletcher is not the same author as cited in [5].<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
== [[Simulated annealing]] ==<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71552024 Cornell Optimization Open Textbook Feedback2024-12-13T17:04:00Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms]] ==<br />
<br />
== [[Local branching]] ==<br />
<br />
== [[Trust-region methods]] ==<br />
<br />
== [[Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming]] ==<br />
<br />
== [[Derivative free optimization]] ==<br />
<br />
== [[Signomial problems]] ==<br />
<br />
== [[Adadelta]] ==<br />
<br />
== [[Adafactor]] ==<br />
<br />
== [[AdamW]] ==<br />
<br />
== [[Adamax]] ==<br />
<br />
== [[FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam]] ==<br />
<br />
== [[Bayesian optimization|Beyesian optimization]] ==<br />
<br />
== [[Genetic algorithm]] ==<br />
<br />
== [[Simulated annealing]] ==<br />
<br />
== [[Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71542024 Cornell Optimization Open Textbook Feedback2024-12-13T17:03:00Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity|Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms|Heuristic algorithms]] ==<br />
<br />
== [[Local branching|Local branching]] ==<br />
<br />
== [[Trust-region methods|Trust-region methods]] ==<br />
<br />
== [[Quadratic programming|Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming|Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization|Subgradient optimization]] ==<br />
<br />
== [[Dynamic optimization|Dynamic optimization]] ==<br />
<br />
== [[Nondifferentiable Optimization|Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization|Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model|Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming|Quadratic constrained quadratic programming]] ==<br />
<br />
== [[Derivative free optimization|Derivative free optimization]] ==<br />
<br />
== [[Signomial problems|Signomial problems]] ==<br />
<br />
== [[Adadelta|Adadelta]] ==<br />
<br />
== [[Adafactor|Adafactor]] ==<br />
<br />
== [[AdamW|AdamW]] ==<br />
<br />
== [[Adamax|Adamax]] ==<br />
<br />
== [[FTRL algorithm|FTRL algorithm]] ==<br />
<br />
== [[Lion algorithm|Lion algorithm]] ==<br />
<br />
== [[LossScaleOptimizer|LossscaleOptimizer]] ==<br />
<br />
== [[Nadam|Nadam]] ==<br />
<br />
== [[Beyesian optimization|Beyesian optimization]] ==<br />
<br />
== [[Genetic algorithm|Genetic algorithm]] ==<br />
<br />
== [[Simulated annealing|Simulated annealing]] ==<br />
<br />
== [[Particle swarm optimization|Particle swarm optimization]] ==<br />
<br />
== [[Differential evolution|Differential evolution]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=2024_Cornell_Optimization_Open_Textbook_Feedback&diff=71532024 Cornell Optimization Open Textbook Feedback2024-12-13T16:55:01Z<p>SYSEN5800TAs: </p>
<hr />
<div>== [[Computational complexity|Computational complexity]] ==<br />
<br />
== [[Heuristic algorithms|Heuristic algorithms]] ==<br />
<br />
== [[Local branching|Local branching]] ==<br />
<br />
== [[Trust-region methods|Trust-region methods]] ==<br />
<br />
== [[Quadratic programming|Quadratic programming]] ==<br />
<br />
== [[Sequential quadratic programming|Sequential quadratic programming]] ==<br />
<br />
== [[Subgradient optimization|Subgradient optimization]] ==<br />
<br />
== [[Dynamic programming|Dynamic programming]] ==<br />
<br />
== [[Nondifferentiable Optimization|Nondifferentiable Optimization]] ==<br />
<br />
== [[Evolutionary multimodal optimization|Evolutionary multimodal optimization]] ==<br />
<br />
== [[Stackelberg leadership model|Stackelberg leadership model]] ==<br />
<br />
== [[Quadratic constrained quadratic programming|Quadratic constrained quadratic programming]] ==<br />
<br />
== [[Derivative free optimization|Derivative free optimization]] ==<br />
<br />
== [[Signomial problems|Signomial problems]] ==<br />
<br />
== [[Adadelta|Adadelta]] ==<br />
<br />
== [[Adafactor|Adafactor]] ==</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=File:Value7000.0.png&diff=6956File:Value7000.0.png2024-12-11T19:34:37Z<p>SYSEN5800TAs: </p>
<hr />
<div>xx</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=Trust-region_methods&diff=6851Trust-region methods2024-12-10T18:57:34Z<p>SYSEN5800TAs: </p>
<hr />
<div>Autor: Tung Yen Wang (tw565) (CHEME 6800, Fall 2024)<br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu<br />
<br />
=Introduction=<br />
<br />
Trust region method is a numerical optimization method that is employed to solve non-linear programming (NLP) problems<ref>Boyd, S., Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization, ''Cambridge university press'', <nowiki>https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf</nowiki></ref>. Instead of finding an objective solution of the original function, the method defines a neighborhood around the current best solution as a trust region in each step (typically by using a quadratic model), which is capable of representing the function appropriately, to derive the next local optimum. Different from line search<ref>Nocedal, J., & Wright, S. (2006). Numerical optimization, ''Springer Science & Business Media'', <nowiki>https://www.math.uci.edu/~qnie/Publications/NumericalOptimization.pdf</nowiki></ref>, the model selects the direction and step size simultaneously. For example, in a minimization problem, if the decrease in the value of the optimal solution is not sufficient since the region is too large to get close to the minimizer of the objective function, the region should be shrunk to find the next best point. On the other hand, if such a decrease is significant, it is believed that the model has an adequate representation of the problem. Generally, the step direction depends on the extent that the region is altered in the previous iteration<ref>Erway, J. B., Gill, P. E., & Griffin, J. D. (2009). Iterative methods for finding a trust-region step, ''SIAM Journal on Optimization'', <nowiki>https://www.math.uci.edu/~qnie/Publications/NumericalOptimization.pdf</nowiki></ref>.<br />
=Methodology and theory=<br />
<br />
The quadratic model function <math>m_k</math> is based on derivative information at <math>x_k</math> and possibly also on information accumulated from previous iterations and steps.<br />
<br />
<math>m_{k}(p)=f_{k} +\bigtriangledown f_k^Tp + 1/2p^TB_kp</math><br />
<br />
where <br />
<br />
<math>f_k=f(x_k), \bigtriangledown f_k=\bigtriangledown f(x_k)</math>, and <math>B_k</math> is symmetric matrix.<br />
<br />
Taylor’s theorem is used as a mathematical tool to study minimizers of smooth functions.<br />
<br />
<math>f(x_k + p)=f_{k} +\bigtriangledown f_k^Tp + 1/2p^T\bigtriangledown ^2f(x_k + tp)p</math><br />
<br />
The first two terms of <math>m_k</math> are assumed to be identical to the first two terms of the Taylor-series expansion.<br />
<br />
<math>m_{k}(p)=f_{k} +\bigtriangledown f_k^Tp + O(||p^2||)</math><br />
<br />
The difference between <math>m_k(p)</math> and <math>f (x_k + p)</math> is O. Therefore, when <math>p</math> is small, the approximation error is small.<br />
<br />
To obtain each step, we seek a solution to the subproblem as shown below.<br />
<br />
<math>\min_{p\in\R} m_{k}(p)=f_{k} +\bigtriangledown f_k^Tp + 1/2p^TB_kp</math><br />
<br />
s.t <math>||p^2||\leqq\Delta _k</math><br />
<br />
The strategies for finding approximate solutions are introduced as follows, which achieve at least as so-called Cauchy point. This point is simply the minimizer of <math>m_k</math> along the steepest descent direction that is subject to the trust region bound.<br />
<br />
'''Cauchy point calculation'''<br />
<br />
Similar to the line search method which does not require optimal step lengths to be convergent, the trust-region method is sufficient for global convergence purposes to find an approximate solution <math>p_k</math> that lies within the trust region. Cauchy step <math>p_k^c</math> is an inexpensive method( no matrix factorization) to solve trust-region subproblem. Furthermore, the Cauchy point has been valued because it can be globally convergent. Following is a closed-form equation of the Cauchy point.<br />
<br />
<math>p_k^c=-\tau _k\frac{\Delta k}{\left \| \bigtriangledown f_k \right \|}\bigtriangledown f_k</math><br />
<br />
where<br />
<br />
<math>\displaystyle \tau _k={\begin{cases}1,&{\text{if }}\bigtriangledown f_k^TB_k\bigtriangledown f_k\leq 0\\min\left ( \left \| \bigtriangledown f_k \right \|^{3}/\left ( \bigtriangleup _k\bigtriangledown f_k^TB_k\bigtriangledown f_k \right ),1 \right ),&{\text{otherwise }}\end{cases}}</math><br />
<br />
<br />
Although it is inexpensive to apply the Cauchy point, the steepest descent methods sometimes perform poorly. Thus, we introduce some improving strategies. The improvement strategies is based on <math>B_k</math> where it contains valid curvature information about the function.<br />
<br />
'''Dogleg method'''<br />
<br />
This method can be used if <math>B_k</math> is a positive definite. The dogleg method finds an approximate solution by replacing the curved trajectory<br />
<br />
for <math>p^*\left ( \bigtriangleup \right )</math> with a path consisting of two line segments. It chooses p to minimize the model m along this path, subject to the trust-region bound.<br />
<br />
First line segments <math>p^U=-\frac{g^Tg}{g^TBg}g </math>, where <math>p^U</math>runs from the origin to the minimizer of m along the steepest descent direction.<br />
<br />
While the second line segment run from <math>p^U</math>to <math>p^B</math>, we donate this trajectory by <math>\tilde{p}\left ( \tau \right )</math> for <math>\tau \in \left [ 0,2 \right ]</math><br />
<br />
Then a V-shaped trajectory can be determined by <br />
<br />
<math>\tilde{p}=\tau p^U </math>, when <math>0\leq \tau \leq 1 </math><br />
<br />
<math>\tilde{p}= p^U+\left (\tau -1 \right )\left ( p^B-p^U \right ) </math>, when <math>1\leq \tau \leq 2 </math><br />
<br />
where <math>p^B </math>=opitimal solution of quadratic model <br />
<br />
Although the dogleg strategy can be adapted to handle indefinite B, there is not much point in doing so because the full step <math>p^B</math> is not the unconstrained minimizer of m in this case. Instead, we now describe another strategy, which aims to include directions of negative<br />
<br />
curvature in the space of trust-region steps.<br />
<br />
<br />
'''Conjugated Gradient Steihaug’s Method ( CG-Steihaug)'''<br />
<br />
This is the most widely used method for the approximate solution of the trust-region problem. The method works for quadratic models <math>m_{k}</math> defined by an arbitrary symmetric <math>B_k</math> . Thus, it is not necessary for <math>B_k</math> to be positive. CG-Steihaug has the advantage of Cauchy point calculation and Dogleg method which is super-linear convergence rate and inexpensive costs.<br />
<br />
Given<math>\epsilon > 0 </math><br />
<br />
Set <math>p_0=0,r_0=g,d_0=-r_0 </math><br />
<br />
if<math>\left \| r_0 \right \|< \epsilon </math><br />
<br />
return <math>p=p0</math><br />
<br />
for <math>j=0,1,2,...</math><br />
<br />
if <math>d_j^TB_kd_j\leq 0 </math><br />
<br />
Find <math>\tau </math> such that minimizes <math>m\left ( p \right )</math> and satisfies<math>\left \| p \right \|=\Delta </math><br />
<br />
return p;<br />
<br />
Set <math>\alpha _j=r_j^Tr_j/d_j^TB_kd_j</math><br />
<br />
Set <math> p_{j+1}=p_j+\alpha _jd_j</math><br />
<br />
if <math>\left \| p_{j+1} \right \|\geq \Delta</math><br />
<br />
Find <math>\tau \geq 0</math> such that <math>p=p_{j}+\tau d_{j}</math> satisfies <math>\left \| p \right \|=\Delta </math><br />
<br />
return p;<br />
<br />
Set <math>r_{j+1}=r_j+\alpha _jB_kd_j</math><br />
<br />
if <math> \left \| r_{j+1} \right \|< \epsilon \left \| r_{0} \right \|</math><br />
<br />
return <math>p=p_{j+1}</math><br />
<br />
Set <math>\beta _{j+1} = r_{j+1}^{T}r_{j+1}/r_j^Tr_j</math><br />
<br />
Set <math>d_{j+1}= r_{j+1}+ \beta _{j+1}d_j</math><br />
<br />
end(for)<br />
<br />
<br />
'''Global Convergence'''<br />
<br />
To study the convergence of trust region, we have to study how much reduction can we achieve at each<br />
<br />
iteration (similar to line search method). Thus, we derive an estimate in the following form:<br />
<br />
<math>m_k\left ( 0 \right )-m_k\left ( p_k \right )\geq c_1\left \| \bigtriangledown f_k \right \|min\left ( \bigtriangleup k,\frac{\left \| \bigtriangledown f_k\right \|}{\left \| B_k \right \|} \right )</math> for <math>c_1\in \left [ 0,1 \right ]</math><br />
<br />
For cauchy point, <math>c_1</math>=0.5<br />
<br />
that is<br />
<br />
<math>m_k\left ( 0 \right )-m_k\left ( p_k \right )\geq 0.5\left \| \bigtriangledown f_k \right \|min\left ( \bigtriangleup k,\frac{\left \| \bigtriangledown f_k\right \|}{\left \| B_k \right \|} \right )</math><br />
<br />
we first consider the case of <math>\bigtriangledown f_k^TB_k\bigtriangledown f_k\leq 0</math><br />
<br />
<math>m_k\left ( p_k^c \right )-m_k\left ( 0 \right )\geq m_k\left ( \bigtriangleup _k\bigtriangledown f_k/\left \| \bigtriangledown f_k \right \| \right )</math><br />
<br />
<math>=-\frac{\bigtriangleup _k}{\left \| \bigtriangledown f_k \right \|}\left \| \bigtriangledown f_k \right \|^2+0.5\frac{\bigtriangleup _k^2}{\left \| \bigtriangledown f_k \right \|^2}\ \bigtriangledown f_k^TB_k\bigtriangledown f_k</math><br />
<br />
<math>\leq -\bigtriangleup _k\left \| \bigtriangledown f_k \right \|</math><br />
<br />
<math>\leq -\left \| \bigtriangledown f_k \right \|min\left ( \bigtriangleup _k,\frac{\left \| \bigtriangledown f_k \right \|}{B_k} \right )</math><br />
<br />
For the next case, consider <math>\bigtriangledown f_k^TB_k\bigtriangledown f_k> 0</math> and<br />
<br />
<math>\frac{\left \| \bigtriangledown f_k \right \|^3}{\bigtriangleup _k\bigtriangledown f_k^TB_k\bigtriangledown f_k}\leq 1</math><br />
<br />
we then have <math>\tau =\frac{\left \| \bigtriangledown f_k \right \|^3}{\bigtriangleup _k\bigtriangledown f_k^TB_k\bigtriangledown f_k}</math><br />
<br />
so<br />
<br />
<math>m_k\left ( p_k^c \right )-m_k\left ( 0 \right )= -\frac{\left \| \bigtriangledown f_k \right \|^4}{\bigtriangledown f_k^TB_k\bigtriangledown f_k}+0.5\bigtriangledown f_k^TB_k\bigtriangledown f_k\frac{\left \| \bigtriangledown f_k \right \|^4}{\left ( \bigtriangledown f_k^TB_k\bigtriangledown f_k \right )^2}</math><br />
<br />
<math>=-0.5\frac{\left \| \bigtriangledown f_k \right \|^4}{\bigtriangledown f_k^TB_k\bigtriangledown f_k}</math><br />
<br />
<math>\leq -0.5\frac{\left \| \bigtriangledown f_k \right \|^4}{\left \| B_k \right \|\left \| \bigtriangledown f_k \right \|^2}</math><br />
<br />
<math>=-0.5\frac{\left \| \bigtriangledown f_k \right \|^2}{\left \| B_k \right \|}</math><br />
<br />
<math>\leq -0.5\left \| \bigtriangledown f_k \right \|min\left ( \bigtriangleup _k,\frac{\left \| \bigtriangledown f_k \right \|}{\left \| B_k \right \|} \right )</math>,<br />
<br />
since <math>\frac{\left \| \bigtriangledown f_k \right \|^3}{\bigtriangleup _k\bigtriangledown f_k^TB_k\bigtriangledown f_k}\leq 1</math> does not hold, thus<br />
<br />
<math>\bigtriangledown f_k^TB_k\bigtriangledown f_k< \frac{\left \| \bigtriangledown f_k \right \|^3}{\bigtriangleup _k}</math><br />
<br />
From the definition of <math>p_c^k</math> , we have <math>\tau =1</math>, therefore<br />
<br />
<math>m_k\left ( p_k^c \right )-m_k\left ( 0 \right )= -\frac{\bigtriangleup _k}{\left \| \bigtriangledown f_k \right \|}\left \| \bigtriangledown f_k \right \|^2+0.5\frac{\bigtriangleup _k^2}{\left \| \bigtriangledown f_k \right \|^2}\bigtriangledown f_k^TB_k\bigtriangledown f_k</math><br />
<br />
<math>\leq -\bigtriangleup _k\left \| \bigtriangledown f_k \right \|^2+0.5\frac{\bigtriangleup _k^2}{\left \| \bigtriangledown f_k \right \|^2}\frac{\left \| \bigtriangledown f_k \right \|^3}{\bigtriangleup _k}</math><br />
<br />
<math>=-0.5\bigtriangleup _k\left \| \bigtriangledown f_k \right \|</math><br />
<br />
<math>\leq -0.5\left \| \bigtriangledown f_k \right \|min\left ( \bigtriangleup _k,\frac{\left \| \bigtriangledown f_k \right \|}{\left \| B_k \right \|} \right )</math><br />
<br />
<br />
=Numerical example=<br />
<br />
<br />
Here we will use the trust-region method to solve a classic optimization problem, the Rosenbrock function. The Rosenbrock function is a non-convex function, introduced by Howard H. Rosenbrock in 1960<ref>H. H. Rosenbrock, An Automatic Method for Finding the Greatest or Least Value of a Function, ''The Computer Journal'', Volume 3, Issue 3, 1960, Pages 175–184, <nowiki>https://doi.org/10.1093/comjnl/3.3.175</nowiki></ref>, which is often used as a performance test problem for optimization algorithms. This problem is solved using MATLAB's <code>fminunc</code> as the solver, with 'trust-region' as the solving algorithm which uses the preconditioned conjugate method. <br />
<br />
The function is defined by<br />
<br />
<math>\min f(x,y)=100(y-x^2)^2+(1-x)^2</math><br />
<br />
The starting point chosen is <math>x=0</math> <math>y=0<br />
</math>.<br />
[[File:Trust-region method example trajectory.png|thumb|576x576px|Trust-region method trajectory of Rosenbrock function starting from (0,0). The data points represent the optimal solutions after each iteration, ending at iteration number 16 (1,1).]]<br />
<br />
<br />
'''Iteration Process'''<br />
<br />
'''Iteration 1:''' The algorithm starts from the initial point of <math>x=0</math>, <math>y=0<br />
</math>. The Rosenbrock function is visualized with a color coded map. For the first iteration, a full step was taken and the optimal solution (<math>x=0.25</math>, <math>y=0<br />
</math>) within the trust-region is denoted as a red dot. <br />
<br />
<br />
'''Iteration 2:''' Start with '''<math>x=0.25<br />
</math>, <math>y=0<br />
</math>'''. The new iteration gives a good prediction, which increases the trust-region's size. The new optimal solution within the trust-region is '''<math>x=0.263177536<br />
</math>, <math>y=0.061095029<br />
</math>'''.<br />
<br />
<br />
'''Iteration 3:''' Start with '''<math>x=0.263177536<br />
</math>, <math>y=0.061095029<br />
</math>'''. The new iteration gives a poor prediction, which decreases the trust-region's size to improve the model's validity. The new optimal solution within the trust-region is <math>x=0.371151679<br />
</math>, <math>y=0.124075855<br />
<br />
</math>.<br />
<br />
...<br />
<br />
'''Iteration 7:''' Start with '''<math>x=0.765122406<br />
</math>, <math>y=0.560476539<br />
<br />
<br />
</math>'''. The new iteration gives a poor prediction, which decreases the trust-region's size to improve the model's validity. The new optimal solution within the trust-region is '''<math>x=0.804352654<br />
<br />
</math>, ''' <math>y=0.645444179<br />
<br />
<br />
</math>.<br />
<br />
<br />
<br />
'''Iteration 8:''' Start with '''<math>x=0.804352654<br />
<br />
</math>, <math>y=0.645444179<br />
<br />
<br />
</math>'''.The new iteration gives a poor prediction, therefore the current best solution is unchanged and the radius for the trust region is decreased.<br />
<br />
...<br />
<br />
At the 16th iteration, the global optimal solution is found, '''<math>x=1<br />
</math>, <math>y=1<br />
<br />
</math>'''. <br />
<br />
{| class="wikitable"<br />
|+Summary of all iterations<br />
!Iterations<br />
!f(x)<br />
!x<br />
!y<br />
!Norm of step<br />
!First-order optimality<br />
|-<br />
|1<br />
|1 <br />
|0.25<br />
|0<br />
|1<br />
|2<br />
|-<br />
|2<br />
|0.953125<br />
|0.263178<br />
|0.061095<br />
|0.25<br />
|12.5<br />
|-<br />
|3<br />
|0.549578 <br />
|0.371152<br />
|0.124076<br />
|0.0625<br />
|1.63<br />
|-<br />
|4<br />
|0.414158<br />
|0.539493<br />
|0.262714<br />
|0.125<br />
|2.74<br />
|-<br />
|5<br />
|0.292376 <br />
|0.608558<br />
|0.365573<br />
|0.218082<br />
|5.67<br />
|-<br />
|6<br />
|0.155502<br />
|0.765122<br />
|0.560477<br />
|0.123894<br />
|0.954<br />
|-<br />
|7<br />
|0.117347<br />
|0.804353<br />
|0.645444<br />
|0.25<br />
|7.16<br />
|-<br />
|8<br />
|0.0385147<br />
|0.804353<br />
|0.645444<br />
|0.093587<br />
|0.308<br />
|-<br />
|9<br />
|0.0385147<br />
|0.836966<br />
|0.69876<br />
|0.284677<br />
|0.308<br />
|-<br />
|10<br />
|0.0268871<br />
|0.90045<br />
|0.806439<br />
|0.0625<br />
|0.351<br />
|-<br />
|11<br />
|0.0118213<br />
|0.953562<br />
|0.90646<br />
|0.125<br />
|1.38<br />
|-<br />
|12<br />
|0.0029522<br />
|0.983251<br />
|0.9659<br />
|0.113247<br />
|0.983<br />
|-<br />
|13<br />
|0.000358233<br />
|0.99749<br />
|0.994783<br />
|0.066442<br />
|0.313<br />
|-<br />
|14<br />
|1.04121e-05<br />
|0.999902<br />
|0.999799<br />
|0.032202<br />
|0.0759<br />
|-<br />
|15<br />
|1.2959e-08<br />
|1<br />
|1<br />
|0.005565<br />
|0.00213<br />
|-<br />
|16<br />
|2.21873e-14<br />
|1<br />
|1<br />
|0.000224<br />
|3.59E-06<br />
|}<br />
<br />
=Applications=<br />
'''Approach on Newton methods on Riemannian manifold'''<br />
<br />
Absil et. Al (2007) proposed a trust-region approach for improving the Newton method on the Riemannian manifold<ref>Absil, PA., Baker, C. & Gallivan, K(2007). Trust-Region Methods on Riemannian Manifolds, ''Found Comput Math 7'', Page 303–330, <nowiki>https://doi.org/10.1007/s10208-005-0179-9</nowiki></ref>. The trust-region approach optimizes a smooth function on a Riemannian manifold in three ways. First, the exponential mapping is relaxed to general retractions with a view to reducing computational complexity. Second, a trust region approach is applied for both local and global convergence. Third, the trust-region approach allows early stopping of the inner iteration under criteria that preserve the convergence properties of the overall algorithm.<br />
<br />
'''Approach on policy optimization'''<br />
<br />
Schulman et. al (2015) proposed trust-region methods for optimizing stochastic control policies and developed a practical algorithm called Trust Region Policy Optimization (TRPO)<ref>Schulman, J., et al. (2015). Trust region policy optimization, ''International conference on machine learning'', <nowiki>http://proceedings.mlr.press/v37/schulman15</nowiki></ref>. The method is scalable and effective for optimizing large nonlinear policies such as neural networks. It can optimize nonlinear policies with tens of thousands of parameters, which is a major challenge for model-free policy search.<br />
<br />
=Conclusion=<br />
The trusted region is a powerful method that can update the objective function in each step to ensure the model is always getting improved while keeping the previously learned knowledge as the baseline. Unlike line search methods, trust region can be used in non-convex approximate models, making such class of iterative methods more reliable, robust and applicable to ill-conditioned problems<ref>Yuan, Y. X. (2000). A review of trust region algorithms for optimization, ''Iciam conference'', <nowiki>https://doi.org/10.4236/blr.2019.104046</nowiki></ref>. Recently, due to its capability to address large-scale problems<ref>Lin, C. J., Weng, R. C., & Keerthi, S. S. (2008). Trust region Newton method for large-scale logistic regression, ''Journal of Machine Learning Research'', <nowiki>https://www.jmlr.org/papers/volume9/lin08b/lin08b.pdf</nowiki></ref><ref>Rojas, M., Santos, S. A., & Sorensen, D. C. (2008). MATLAB software for large-scale trust-region subproblems and regularization, ''ACM Transactions on Mathematical Software'', <nowiki>https://doi.org/10.1145/1326548.1326553</nowiki></ref><ref>Wu, Y., Mansimov, E., Grosse, R. B., Liao, S., & Ba, J. (2017). Scalable trust-region method for deep reinforcement learning using kronecker-factored approximation, ''Advances in neural information processing systems'', <nowiki>https://proceedings.neurips.cc/paper/2017/file/361440528766bbaaaa1901845cf4152b-Paper.pdf</nowiki></ref>, trust region has been paired with several machine-learning topics, including tuning parameter selection<ref>Geminiani, E., Marra, G., & Moustaki, I. (2021). Single-and Multiple-Group Penalized Factor Analysis: A Trust-Region Algorithm Approach with Integrated Automatic Multiple Tuning Parameter Selection, ''psychometrika'', Page 65-69, <nowiki>https://doi.org/10.1007/s11336-021-09751-8</nowiki></ref>, ridge function<ref>Gross, J. C., Seshadri, P., & Parks, G. (2020). Optimisation with intrinsic dimension reduction: A ridge informed trust-region method, ''AIAA Scitech 2020 Forum'', <nowiki>https://doi.org/10.2514/6.2020-0157</nowiki></ref>, reinforcement learning<ref>Kuba, J. G., Chen, R., Wen, M., Wen, Y., Sun, F., Wang, J., & Yang, Y. (2021). Trust region policy optimisation in multi-agent reinforcement learning, ''arXiv preprint arXiv:2109.11251'', <nowiki>https://arxiv.org/pdf/2109.11251.pdf</nowiki></ref>, etc., to develop more robust numerical algorithms. It is believed that the trust region method will have more far-reaching development in a wider range of fields in the near future.<br />
<br />
=References=<br />
<references /></div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=Local_branching&diff=6850Local branching2024-12-10T18:56:41Z<p>SYSEN5800TAs: </p>
<hr />
<div>Author: Bruce Wang (bw549), Ashley Yang (yy2333), Pufan You (py234), Sandro Xu (sx289), Yihan Wang (yw2744) (ChemE 6800 Fall 2020)<br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=Evolutionary_multimodal_optimization&diff=6849Evolutionary multimodal optimization2024-12-10T18:55:49Z<p>SYSEN5800TAs: </p>
<hr />
<div>Author: Connor Clappin (cjc395) (ChemE 6800 Fall 2024)<br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=Particle_swarm_optimization&diff=6848Particle swarm optimization2024-12-10T18:55:11Z<p>SYSEN5800TAs: </p>
<hr />
<div>Author: David Schluneker (dms565), Thomas Ploetz (tep52), Michael Sorensen (mds385), Amrith Kumaar (ak836), Andrew Duffy (ajd296) (ChemE 6800 Fall 2024)<br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=Differential_evolution&diff=6847Differential evolution2024-12-10T18:53:55Z<p>SYSEN5800TAs: </p>
<hr />
<div>Author: Rohit Kumar (rk787) (ChemE 6800 Fall 2024)<br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=Simulated_annealing&diff=6846Simulated annealing2024-12-10T18:53:18Z<p>SYSEN5800TAs: </p>
<hr />
<div>Author: Gwen Zhang (xz929), Yingjie Wang (yw2749), Junchi Xiao (jx422), Yichen Li (yl3938), Xiaoxiao Ge (xg353) (ChemE 6800 Fall 2024)<br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu</div>SYSEN5800TAshttps://optimization.cbe.cornell.edu/index.php?title=Heuristic_algorithms&diff=6845Heuristic algorithms2024-12-10T18:51:55Z<p>SYSEN5800TAs: </p>
<hr />
<div>Author: Zemin Mi (zm287), Boyu Yang (by274), Jinglin Wang (jw2745) (ChemE 6800 Fall 2024)<br />
<br />
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu<br />
<br />
== Introduction ==<br />
In mathematical programming, a heuristic algorithm is a procedure that determines near-optimal solutions to an optimization problem. However, this is achieved by trading optimality, completeness, accuracy, or precision for speed.<ref> Eiselt, Horst A et al. Integer Programming and Network Models. Springer, 2011.</ref> Nevertheless, heuristics is a widely used technique for a variety of reasons:<br />
<br />
*Problems that do not have an exact solution or for which the formulation is unknown<br />
*The computation of a problem is computationally intensive<br />
*Calculation of bounds on the optimal solution in branch and bound solution processes<br />
==Methodology==<br />
Optimization heuristics can be categorized into two broad classes depending on the way the solution domain is organized:<br />
<br />
===Construction methods (Greedy algorithms)===<br />
The greedy algorithm works in phases, where the algorithm makes the optimal choice at each step as it attempts to find the overall optimal way to solve the entire problem.<ref><br />
''Introduction to Algorithms'' (Cormen, Leiserson, Rivest, and Stein) 2001, Chapter 16 "Greedy Algorithms".</ref> It is a technique used to solve the famous “traveling salesman problem” where the heuristic followed is: "At each step of the journey, visit the nearest unvisited city." <br />
<br />
====Example: Scheduling Problem====<br />
You are given a set of N schedules of lectures for a single day at a university. The schedule for a specific lecture is of the form (s time, f time) where s time represents the start time for that lecture, and similarly, the f time represents the finishing time. Given a list of N lecture schedules, we need to select a maximum set of lectures to be held out during the day such that none of the lectures overlaps with one another i.e. if lecture L<sub>i</sub> and L<sub>j</sub> are included in our selection then the start time of j ≥ finish time of i or vice versa. The most optimal solution to this would be to consider the earliest finishing time first. We would sort the intervals according to the increasing order of their finishing times and then start selecting intervals from the very beginning. <br />
<br />
===Local Search methods===<br />
The Local Search method follows an iterative approach where we start with some initial solution, explore the neighborhood of the current solution, and then replace the current solution with a better solution.<ref> Eiselt, Horst A et al. Integer Programming and Network Models. Springer, 2011.</ref> For this method, the “traveling salesman problem” would follow the heuristic in which a solution is a cycle containing all nodes of the graph and the target is to minimize the total length of the cycle.<br />
<br />
==== Example Problem ====<br />
Suppose that the problem P is to find an optimal ordering of N jobs in a manufacturing system. A solution to this problem can be described as an N-vector of job numbers, in which the position of each job in the vector defines the order in which the job will be processed. For example, [3, 4, 1, 6, 5, 2] is a possible ordering of 6 jobs, where job 3 is processed first, followed by job 4, then job 1, and so on, finishing with job 2. Define now M as the set of moves that produce new orderings by the swapping of any two jobs. For example, [3, 1, 4, 6, 5, 2] is obtained by swapping the positions of jobs 4 and 1.<br />
==Popular Heuristic Algorithms==<br />
<br />
===Genetic Algorithm===<br />
The term Genetic Algorithm was first used by John Holland.<ref>J.H. Holland (1975) ''Adaptation in Natural and Artificial Systems,'' University of Michigan Press, Ann Arbor, Michigan; re-issued by MIT Press (1992).</ref> They are designed to mimic the Darwinian theory of evolution, which states that populations of species evolve to produce more complex organisms and fitter for survival on Earth. Genetic algorithms operate on string structures, like biological structures, which are evolving in time according to the rule of survival of the fittest by using a randomized yet structured information exchange. Thus, in every generation, a new set of strings is created, using parts of the fittest members of the old set.<ref>Optimal design of heat exchanger networks, Editor(s): Wilfried Roetzel, Xing Luo, Dezhen Chen, Design and Operation of Heat Exchangers and their Networks, Academic Press, 2020, Pages 231-317, <nowiki>ISBN 9780128178942</nowiki>, https://doi.org/10.1016/B978-0-12-817894-2.00006-6.</ref> The algorithm terminates when the satisfactory fitness level has been reached for the population or the maximum generations have been reached. The typical steps are<ref>Wang FS., Chen LH. (2013) Genetic Algorithms. In: Dubitzky W., Wolkenhauer O., Cho KH., Yokota H. (eds) Encyclopedia of Systems Biology. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9863-7_412 </ref>:<br />
<br />
1. Choose an initial population of candidate solutions<br />
<br />
2. Calculate the fitness, how well the solution is, of each individual<br />
<br />
3. Perform crossover from the population. The operation is to randomly choose some pair of individuals like parents and exchange so parts from the parents to generate new individuals<br />
<br />
4. Mutation is to randomly change some individuals to create other new individuals<br />
<br />
5. Evaluate the fitness of the offspring<br />
<br />
6. Select the survive individuals<br />
<br />
7. Proceed from 3 if the termination criteria have not been reached<br />
<br />
===Tabu Search Algorithm===<br />
Tabu search (TS) is a heuristic algorithm created by Fred Glover<ref>Fred Glover (1986). "Future Paths for Integer Programming and Links to Artificial Intelligence". Computers and Operations Research. '''13''' (5): 533–549,https://doi.org/10.1016/0305-0548(86)90048-1</ref> using a gradient-descent search with memory techniques to avoid cycling for determining an optimal solution. It does so by forbidding or penalizing moves that take the solution, in the next iteration, to points in the solution space previously visited. The algorithm spends some memory to keep a Tabu list of forbidden moves, which are the moves of the previous iterations or moves that might be considered unwanted. A general algorithm is as follows<ref>Optimization of Preventive Maintenance Program for Imaging Equipment in Hospitals, Editor(s): Zdravko Kravanja, Miloš Bogataj, Computer-Aided Chemical Engineering, Elsevier, Volume 38, 2016, Pages 1833-1838, ISSN 1570-7946, <nowiki>ISBN 9780444634283</nowiki>, https://doi.org/10.1016/B978-0-444-63428-3.50310-6.</ref>: <br />
<br />
1. Select an initial solution ''s''<sub>0</sub> ∈ ''S''. Initialize the Tabu List ''L''<sub>0</sub> = ∅ and select a list tabu size. Establish ''k'' = 0.<br />
<br />
2. Determine the neighborhood feasibility ''N''(''s<sub>k</sub>'') that excludes inferior members of the tabu list ''L<sub>k</sub>''.<br />
<br />
3. Select the next movement ''s<sub>k</sub>'' <sub>+ 1</sub> from ''N''(''S<sub>k</sub>'') or ''L<sub>k</sub>'' if there is a better solution and update ''L<sub>k</sub>'' <sub>+ 1</sub><br />
<br />
4. Stop if a condition of termination is reached, else, ''k'' = ''k'' + 1 and return to 1<br />
<br />
==== Example: The Classical Vehicle Routing Problem ====<br />
''Vehicle Routing Problems'' have very important applications in distribution management and have become some of the most studied problems in the combinatorial optimization literature. These include several Tabu Search implementations that currently rank among the most effective. The ''Classical Vehicle Routing Problem'' (CVRP) is the basic variant in that class of problems. It can formally be defined as follows. Let ''G'' = (''V, A'') be a graph where ''V'' is the vertex set and ''A'' is the arc set. One of the vertices represents the ''depot'' at which a fleet of identical vehicles of capacity ''Q'' is based, and the other vertices customers that need to be serviced. With each customer vertex v<sub>i</sub> are associated a demand q<sub>i</sub> and a service time t<sub>i</sub>. With each arc (v<sub>i</sub>, v<sub>j</sub>) of ''A'' are associated a cost c<sub>ij</sub> and a travel time t<sub>ij</sub>.<ref>Glover, Fred, and Gary A Kochenberger. Handbook Of Metaheuristics. Kluwer Academic Publishers, 2003.</ref> The CVRP consists of finding a set of routes such that:<br />
<br />
1. Each route begins and ends at the depot<br />
<br />
2. Each customer is visited exactly once by exactly one route<br />
<br />
3. The total demand of the customers assigned to each route does not exceed ''Q''<br />
<br />
4. The total duration of each route (including travel and service times) does not exceed a specified value ''L''<br />
<br />
5. The total cost of the routes is minimized<br />
<br />
A feasible solution for the problem thus consists of a partition of the customers into m groups, each of total demand no larger than ''Q'', that are sequenced to yield routes (starting and ending at the depot) of duration no larger than ''L''.<br />
<br />
===Simulated Annealing Algorithm===<br />
The Simulated Annealing Algorithm was developed by Kirkpatrick et. al. in 1983<ref>Kirkpatrick, S., Gelatt, C., & Vecchi, M. (1983). Optimization by Simulated Annealing. ''Science,'' ''220''(4598), 671-680. Retrieved November 25, 2020, from http://www.jstor.org/stable/1690046</ref> and is based on the analogy of ideal crystals in thermodynamics. The annealing process in metallurgy can make particles arrange themselves in the position with minima potential as the temperature is slowly decreased. The Simulation Annealing algorithm mimics this mechanism and uses the objective function of an optimization problem instead of the energy of a material to arrive at a solution. A general algorithm is as follows<ref>Brief review of static optimization methods, Editor(s): Stanisław Sieniutycz, Jacek Jeżowski, Energy Optimization in Process Systems and Fuel Cells (Third Edition), Elsevier, 2018, Pages 1-41, <nowiki>ISBN 9780081025574</nowiki>, https://doi.org/10.1016/B978-0-08-102557-4.00001-3.</ref> :<br />
<br />
1. Fix initial temperature (''T''<sup>0</sup>)<br />
<br />
2. Generate starting point '''x'''<sup>0</sup> (this is the best point '''''X'''''<sup>*</sup> at present)<br />
<br />
3. Generate randomly point '''''X<sup>S</sup>''''' (neighboring point)<br />
<br />
4. Accept '''''X<sup>S</sup>''''' as '''''X'''''<sup>*</sup> (currently best solution) if an acceptance criterion is met. This must be such a condition that the probability of accepting a worse point is greater than zero, particularly at higher temperatures<br />
<br />
5. If an equilibrium condition is satisfied, go to (6), otherwise jump back to (3).<br />
<br />
6. If termination conditions are not met, decrease the temperature according to a certain cooling scheme and jump back to (1). If the termination conditions are satisfied, stop calculations accepting the current best value '''''X'''''<sup>*</sup> as the final (‘optimal’) solution. <br />
<br />
== Numerical Example: Knapsack Problem ==<br />
One of the most common applications of the heuristic algorithm is the Knapsack Problem, in which a given set of items (each with a mass and a value) are grouped to have a maximum value while being under a certain mass limit. It uses the Greedy Approximation Algorithm to sort the items based on their value per unit mass and then includes the items with the highest value per unit mass if there is still space remaining.<br />
<br />
'''<big>Example</big>'''<br />
<br />
The following table specifies the weights and values per unit of five different products held in storage. The quantity of each product is unlimited. A plane with a weight capacity of 13 is to be used, for one trip only, to transport the products. We would like to know how many units of each product should be loaded onto the plane to maximize the value of goods shipped. <br />
{| class="wikitable"<br />
|+<br />
!<br />
Product (i) <br />
!Weight per unit (w<sub>i</sub>)<br />
!Value per unit (v<sub>i</sub>)<br />
|-<br />
|1<br />
|7<br />
|9<br />
|-<br />
|2<br />
|5<br />
|4<br />
|-<br />
|3<br />
|4<br />
|3<br />
|-<br />
|4<br />
|3<br />
|2<br />
|-<br />
|5<br />
|1<br />
|0.5<br />
|}<br />
'''<big>Solution:</big>'''<br />
<br />
'''(a) Stages:'''<br />
<br />
We view each type of product as a stage, so there are 5 stages. We can also add a sixth stage representing the endpoint after deciding<br />
<br />
'''(b) States:'''<br />
<br />
We can view the remaining capacity as states, so there are 14 states in each stage: 0,1, 2, 3, …13<br />
<br />
'''(c) Possible decisions at each stage:'''<br />
<br />
Suppose we are in state s in stage n (n < 6), hence there are s capacity remaining. Then the possible number of items we can pack is:<br />
<br />
j = 0, 1, …[s/w<sub>n</sub>]<br />
<br />
For each such action j, we can have an arc going from the state s in stage n to the state n – j*w<sub>n</sub> in stage n + 1. For each arc in the graph, there is a corresponding benefit j*v<sub>n</sub>. We are trying to find a maximum benefit path from state 13 in stage 1, to stage 6.<br />
<br />
'''(d) Optimization function:'''<br />
<br />
Let f<sub>n</sub>(s) be the value of the maximum benefit possible with items of type n or greater using total capacity at most s<br />
<br />
'''(e) Boundary conditions:'''<br />
<br />
The sixth stage should have all zeros, that is, f<sub>6</sub>(s) = 0 for each s = 0,1, … 13<br />
<br />
'''(f) Recurrence relation:'''<br />
<br />
f<sub>n</sub>(s) = max {j*v<sub>n</sub> + f<sub>n+1</sub>(s – j*w<sub>n</sub>)}, j = 0, 1, …, [s/w<sub>n</sub>]<br />
<br />
'''(g) Compute:'''<br />
<br />
The solution will not show all the computations steps. Instead, only a few cases are given below to illustrate the idea.<br />
<br />
* For stage 5, f<sub>5</sub>(s) = max<sub>j=0, 1, …[s/1]</sub> {j*0.5 + 0} = 0.5s because given the all zero states in stage 6, the maximum possible value is to use up all the remaining s capacity.<br />
* For stage 4, state 7,<br />
<br />
f<sub>4</sub>(7) = max<sub>j=0,1, …, [7/w4]</sub> = {j*v<sub>4</sub> + f<sub>5</sub>(7 - w<sub>4*</sub>j)}<br />
<br />
= max {0 + 3.5; 2 + 2; 4 + 0.5}<br />
<br />
= 4.5<br />
<br />
Using the recurrence relation above, we get the following table:<br />
{| class="wikitable"<br />
|+<br />
!Unused Capacity<br />
s<br />
!f<sub>1</sub>(s)<br />
!Type 1 <br />
opt<br />
!f<sub>2</sub>(s)<br />
!Type 2 <br />
opt<br />
!f<sub>3</sub>(s)<br />
!Type 3 <br />
opt<br />
!f<sub>4</sub>(s)<br />
!Type 4 <br />
opt<br />
!f<sub>5</sub>(s)<br />
!Type 5 <br />
opt<br />
!f<sub>6</sub>(s)<br />
|-<br />
|13<br />
|13.5<br />
|1<br />
|10<br />
|2<br />
|9.5<br />
|3<br />
|8.5<br />
|4<br />
|6.5<br />
|13<br />
|0<br />
|-<br />
|12<br />
|13<br />
|1<br />
|9<br />
|2<br />
|9<br />
|3<br />
|8<br />
|4<br />
|6<br />
|12<br />
|0<br />
|-<br />
|11<br />
|12<br />
|1<br />
|8.5<br />
|2<br />
|8<br />
|2<br />
|7<br />
|3<br />
|5.5<br />
|11<br />
|0<br />
|-<br />
|10<br />
|11<br />
|1<br />
|8<br />
|2<br />
|7<br />
|2<br />
|6.5<br />
|3<br />
|5<br />
|10<br />
|0<br />
|-<br />
|9<br />
|10<br />
|1<br />
|7<br />
|1<br />
|6.5<br />
|2<br />
|6<br />
|3<br />
|4.5<br />
|9<br />
|0<br />
|-<br />
|8<br />
|9.5<br />
|1<br />
|6<br />
|1<br />
|6<br />
|2<br />
|5<br />
|2<br />
|4<br />
|8<br />
|0<br />
|-<br />
|7<br />
|9<br />
|1<br />
|5<br />
|1<br />
|5<br />
|1<br />
|4.5<br />
|2<br />
|3.5<br />
|7<br />
|0<br />
|-<br />
|6<br />
|4.5<br />
|0<br />
|4.5<br />
|1<br />
|4<br />
|1<br />
|4<br />
|2<br />
|3<br />
|6<br />
|0<br />
|-<br />
|5<br />
|4<br />
|0<br />
|4<br />
|1<br />
|3.5<br />
|1<br />
|3<br />
|1<br />
|2.5<br />
|5<br />
|0<br />
|-<br />
|4<br />
|3<br />
|0<br />
|3<br />
|0<br />
|3<br />
|1<br />
|2.5<br />
|1<br />
|2<br />
|4<br />
|0<br />
|-<br />
|3<br />
|2<br />
|0<br />
|2<br />
|0<br />
|2<br />
|0<br />
|2<br />
|1<br />
|1.5<br />
|3<br />
|0<br />
|-<br />
|2<br />
|1<br />
|0<br />
|1<br />
|0<br />
|1<br />
|0<br />
|1<br />
|0<br />
|1<br />
|2<br />
|0<br />
|-<br />
|1<br />
|0.5<br />
|0<br />
|0.5<br />
|0<br />
|0.5<br />
|0<br />
|0.5<br />
|0<br />
|0.5<br />
|1<br />
|0<br />
|-<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|0<br />
|}<br />
'''Optimal solution:''' The maximum benefit possible is 13.5. Tracing forward to get the optimal solution: the optimal decision corresponding to the entry 13.5 for f<sub>1</sub>(1) is 1, therefore we should pack 1 unit of type 1. After that we have 6 capacity remaining, so look at f<sub>2</sub>(6) which is 4.5, corresponding to the optimal decision of packing 1 unit of type 2. After this, we have 6-5 = 1 capacity remaining, and f<sub>3</sub>(1) = f<sub>4</sub>(1) = 0, which means we are not able to pack any type 3 or type 4. Hence we go to stage 5 and find that f<sub>5</sub>(1) = 1, so we should pack 1 unit of type 5. This gives the entire optimal solution as can be seen in the table below:<br />
{| class="wikitable"<br />
|+<br />
! colspan="2" |Optimal solution<br />
|-<br />
!Product (i)<br />
!Number of units<br />
|-<br />
|1<br />
|1<br />
|-<br />
|2<br />
|1<br />
|-<br />
|5<br />
|1<br />
|}<br />
<br />
==Applications==<br />
Heuristic algorithms have become an important technique in solving current real-world problems. Its applications can range from optimizing the power flow in modern power systems<ref> NIU, M., WAN, C. & Xu, Z. A review on applications of heuristic optimization algorithms for optimal power flow in modern power systems. J. Mod. Power Syst. Clean Energy 2, 289–297 (2014), https://doi.org/10.1007/s40565-014-0089-4</ref> to groundwater pumping simulation models<ref> J. L. Wang, Y. H. Lin and M. D. Lin, "Application of heuristic algorithms on groundwater pumping source identification problems," 2015 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), Singapore, 2015, pp. 858-862, https://doi.org/10.1109/IEEM.2015.7385770.</ref>. Heuristic optimization techniques are increasingly applied in environmental engineering applications as well such as the design of a multilayer sorptive barrier system for landfill liner.<ref>Matott, L. Shawn, et al. “Application of Heuristic Optimization Techniques and Algorithm Tuning to Multilayered Sorptive Barrier Design.” Environmental Science &amp; Technology, vol. 40, no. 20, 2006, pp. 6354–6360., https://doi.org/10.1021/es052560+.</ref> Heuristic algorithms have also been applied in the fields of bioinformatics, computational biology, and systems biology.<ref>Larranaga P, Calvo B, Santana R, Bielza C, Galdiano J, Inza I, Lozano JA, Armananzas R, Santafe G, Perez A, Robles V (2006) Machine learning in bioinformatics. Brief Bioinform 7(1):86–112 </ref><br />
<br />
==Conclusion==<br />
Heuristic algorithms are not a panacea, but they are handy tools to be used when the use of exact methods cannot be implemented. Heuristics can provide flexible techniques to solve hard problems with the advantage of simple implementation and low computational cost. Over the years, we have seen a progression in heuristics with the development of hybrid systems that combine selected features from various types of heuristic algorithms such as tabu search, simulated annealing, and genetic or evolutionary computing. Future research will continue to expand the capabilities of existing heuristics to solve complex real-world problems.<br />
<br />
==References==<br />
<references /></div>SYSEN5800TAs